[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250112 20:56:32.754454 15454 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.23.190:42983
I20250112 20:56:32.755812 15454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250112 20:56:32.756836 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:56:32.773067 15460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:56:32.775420 15463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:56:32.773909 15461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:56:32.776293 15454 server_base.cc:1034] running on GCE node
I20250112 20:56:33.956125 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:56:33.956352 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:56:33.956542 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715393956515 us; error 0 us; skew 500 ppm
I20250112 20:56:33.957387 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:56:33.967377 15454 webserver.cc:458] Webserver started at http://127.15.23.190:39005/ using document root <none> and password file <none>
I20250112 20:56:33.968228 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:56:33.968432 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:56:33.968856 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:56:33.974499 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/master-0-root/instance:
uuid: "0c7eec2e6c784725bdaec7ec25aaa4af"
format_stamp: "Formatted at 2025-01-12 20:56:33 on dist-test-slave-m2nm"
I20250112 20:56:33.981880 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20250112 20:56:33.986527 15469 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:33.987557 15454 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250112 20:56:33.987861 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/master-0-root
uuid: "0c7eec2e6c784725bdaec7ec25aaa4af"
format_stamp: "Formatted at 2025-01-12 20:56:33 on dist-test-slave-m2nm"
I20250112 20:56:33.988198 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:56:34.045064 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:56:34.046382 15454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250112 20:56:34.046823 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:56:34.110764 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.190:42983
I20250112 20:56:34.110837 15520 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.190:42983 every 8 connection(s)
I20250112 20:56:34.117095 15521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:34.138943 15521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af: Bootstrap starting.
I20250112 20:56:34.144248 15521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:34.145923 15521 log.cc:826] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af: Log is configured to *not* fsync() on all Append() calls
I20250112 20:56:34.149870 15521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af: No bootstrap required, opened a new log
I20250112 20:56:34.170769 15521 raft_consensus.cc:357] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7eec2e6c784725bdaec7ec25aaa4af" member_type: VOTER }
I20250112 20:56:34.171296 15521 raft_consensus.cc:383] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:34.171504 15521 raft_consensus.cc:738] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7eec2e6c784725bdaec7ec25aaa4af, State: Initialized, Role: FOLLOWER
I20250112 20:56:34.172189 15521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7eec2e6c784725bdaec7ec25aaa4af" member_type: VOTER }
I20250112 20:56:34.172624 15521 raft_consensus.cc:397] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 20:56:34.172856 15521 raft_consensus.cc:491] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 20:56:34.173125 15521 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:34.176584 15521 raft_consensus.cc:513] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7eec2e6c784725bdaec7ec25aaa4af" member_type: VOTER }
I20250112 20:56:34.177237 15521 leader_election.cc:304] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c7eec2e6c784725bdaec7ec25aaa4af; no voters:
I20250112 20:56:34.178828 15521 leader_election.cc:290] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 20:56:34.179183 15524 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:34.181226 15524 raft_consensus.cc:695] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 1 LEADER]: Becoming Leader. State: Replica: 0c7eec2e6c784725bdaec7ec25aaa4af, State: Running, Role: LEADER
I20250112 20:56:34.182082 15524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7eec2e6c784725bdaec7ec25aaa4af" member_type: VOTER }
I20250112 20:56:34.183108 15521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [sys.catalog]: configured and running, proceeding with master startup.
I20250112 20:56:34.193449 15526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c7eec2e6c784725bdaec7ec25aaa4af. Latest consensus state: current_term: 1 leader_uuid: "0c7eec2e6c784725bdaec7ec25aaa4af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7eec2e6c784725bdaec7ec25aaa4af" member_type: VOTER } }
I20250112 20:56:34.193581 15525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c7eec2e6c784725bdaec7ec25aaa4af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7eec2e6c784725bdaec7ec25aaa4af" member_type: VOTER } }
I20250112 20:56:34.194242 15526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [sys.catalog]: This master's current role is: LEADER
I20250112 20:56:34.194337 15525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [sys.catalog]: This master's current role is: LEADER
I20250112 20:56:34.197558 15534 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 20:56:34.206405 15534 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 20:56:34.208036 15454 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250112 20:56:34.220955 15534 catalog_manager.cc:1348] Generated new cluster ID: fb74b7a66c5945f9a0eacf8dc525768e
I20250112 20:56:34.221230 15534 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 20:56:34.251114 15534 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 20:56:34.253151 15534 catalog_manager.cc:1505] Loading token signing keys...
I20250112 20:56:34.276980 15534 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af: Generated new TSK 0
I20250112 20:56:34.277812 15534 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 20:56:34.338757 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:56:34.347579 15543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:56:34.348723 15544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:56:34.352824 15454 server_base.cc:1034] running on GCE node
W20250112 20:56:34.353766 15547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:56:34.354571 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:56:34.354770 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:56:34.354954 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715394354936 us; error 0 us; skew 500 ppm
I20250112 20:56:34.355474 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:56:34.359854 15454 webserver.cc:458] Webserver started at http://127.15.23.129:35717/ using document root <none> and password file <none>
I20250112 20:56:34.360347 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:56:34.360525 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:56:34.360863 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:56:34.361968 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-0-root/instance:
uuid: "546138398ea14db0bcfe85636a6eebe1"
format_stamp: "Formatted at 2025-01-12 20:56:34 on dist-test-slave-m2nm"
I20250112 20:56:34.367509 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20250112 20:56:34.371670 15551 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:34.372411 15454 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250112 20:56:34.372690 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-0-root
uuid: "546138398ea14db0bcfe85636a6eebe1"
format_stamp: "Formatted at 2025-01-12 20:56:34 on dist-test-slave-m2nm"
I20250112 20:56:34.372960 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:56:34.391398 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:56:34.392561 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:56:34.395115 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:56:34.399242 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:56:34.399444 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:34.399708 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:56:34.399868 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:34.500330 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.129:44501
I20250112 20:56:34.500434 15613 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.129:44501 every 8 connection(s)
I20250112 20:56:34.509655 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:56:34.519510 15618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:56:34.523407 15619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:56:34.531997 15454 server_base.cc:1034] running on GCE node
W20250112 20:56:34.533388 15622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:56:34.534436 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:56:34.534694 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:56:34.534907 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715394534890 us; error 0 us; skew 500 ppm
I20250112 20:56:34.535135 15614 heartbeater.cc:346] Connected to a master server at 127.15.23.190:42983
I20250112 20:56:34.535466 15614 heartbeater.cc:463] Registering TS with master...
I20250112 20:56:34.535638 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:56:34.536294 15614 heartbeater.cc:510] Master 127.15.23.190:42983 requested a full tablet report, sending...
I20250112 20:56:34.538959 15454 webserver.cc:458] Webserver started at http://127.15.23.130:45989/ using document root <none> and password file <none>
I20250112 20:56:34.539476 15486 ts_manager.cc:194] Registered new tserver with Master: 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501)
I20250112 20:56:34.539660 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:56:34.539944 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:56:34.540279 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:56:34.541558 15486 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42770
I20250112 20:56:34.541700 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-1-root/instance:
uuid: "3105649bf63c47b6b4c11216164d816f"
format_stamp: "Formatted at 2025-01-12 20:56:34 on dist-test-slave-m2nm"
I20250112 20:56:34.548274 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20250112 20:56:34.553200 15626 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:34.554020 15454 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250112 20:56:34.554332 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-1-root
uuid: "3105649bf63c47b6b4c11216164d816f"
format_stamp: "Formatted at 2025-01-12 20:56:34 on dist-test-slave-m2nm"
I20250112 20:56:34.554666 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:56:34.577015 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:56:34.578634 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:56:34.580992 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:56:34.584540 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:56:34.584807 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250112 20:56:34.585072 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:56:34.585266 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:34.667151 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.130:42339
I20250112 20:56:34.667234 15688 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.130:42339 every 8 connection(s)
I20250112 20:56:34.675174 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:56:34.685171 15692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:56:34.686156 15689 heartbeater.cc:346] Connected to a master server at 127.15.23.190:42983
I20250112 20:56:34.686499 15689 heartbeater.cc:463] Registering TS with master...
W20250112 20:56:34.687448 15693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:56:34.687526 15689 heartbeater.cc:510] Master 127.15.23.190:42983 requested a full tablet report, sending...
I20250112 20:56:34.689879 15486 ts_manager.cc:194] Registered new tserver with Master: 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
W20250112 20:56:34.690981 15695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:56:34.691552 15486 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42780
I20250112 20:56:34.691874 15454 server_base.cc:1034] running on GCE node
I20250112 20:56:34.692802 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:56:34.693042 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:56:34.693240 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715394693222 us; error 0 us; skew 500 ppm
I20250112 20:56:34.693797 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:56:34.696457 15454 webserver.cc:458] Webserver started at http://127.15.23.131:37049/ using document root <none> and password file <none>
I20250112 20:56:34.696913 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:56:34.697113 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:56:34.697343 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:56:34.698387 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-2-root/instance:
uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
format_stamp: "Formatted at 2025-01-12 20:56:34 on dist-test-slave-m2nm"
I20250112 20:56:34.702478 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20250112 20:56:34.705472 15700 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:34.706249 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250112 20:56:34.706490 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-2-root
uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
format_stamp: "Formatted at 2025-01-12 20:56:34 on dist-test-slave-m2nm"
I20250112 20:56:34.706741 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1736715392595610-15454-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:56:34.725418 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:56:34.726424 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:56:34.727861 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:56:34.730393 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:56:34.730569 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:34.730775 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:56:34.730914 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:56:34.815601 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.131:38129
I20250112 20:56:34.816166 15762 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.131:38129 every 8 connection(s)
I20250112 20:56:34.830524 15763 heartbeater.cc:346] Connected to a master server at 127.15.23.190:42983
I20250112 20:56:34.830907 15763 heartbeater.cc:463] Registering TS with master...
I20250112 20:56:34.831753 15763 heartbeater.cc:510] Master 127.15.23.190:42983 requested a full tablet report, sending...
I20250112 20:56:34.833628 15486 ts_manager.cc:194] Registered new tserver with Master: f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:34.833832 15454 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013765135s
I20250112 20:56:34.835017 15486 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53746
I20250112 20:56:34.837571 15454 test_util.cc:274] Using random seed: -1875940313
I20250112 20:56:34.888336 15486 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53760:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250112 20:56:34.893290 15486 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250112 20:56:34.988289 15579 tablet_service.cc:1467] Processing CreateTablet for tablet 5984e78e6a014ec6951f87f52ca0c5e4 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:56:34.989081 15578 tablet_service.cc:1467] Processing CreateTablet for tablet 46abc6e120e3421e80afc6dd0d5d84c9 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:56:34.989867 15579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5984e78e6a014ec6951f87f52ca0c5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:34.990511 15578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46abc6e120e3421e80afc6dd0d5d84c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:34.993813 15577 tablet_service.cc:1467] Processing CreateTablet for tablet 78a37d408cd1479e93d53989431583f3 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:56:34.995239 15577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a37d408cd1479e93d53989431583f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:34.993893 15728 tablet_service.cc:1467] Processing CreateTablet for tablet 5984e78e6a014ec6951f87f52ca0c5e4 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:56:34.996539 15727 tablet_service.cc:1467] Processing CreateTablet for tablet 46abc6e120e3421e80afc6dd0d5d84c9 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:56:34.996249 15726 tablet_service.cc:1467] Processing CreateTablet for tablet 78a37d408cd1479e93d53989431583f3 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:56:35.001026 15727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46abc6e120e3421e80afc6dd0d5d84c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.005970 15728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5984e78e6a014ec6951f87f52ca0c5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.002337 15575 tablet_service.cc:1467] Processing CreateTablet for tablet 355ac9e86ce94946bb77e763ce9c0731 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:56:35.001140 15576 tablet_service.cc:1467] Processing CreateTablet for tablet a08aeabd9c134e689630f7828f0c4141 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:56:35.011006 15574 tablet_service.cc:1467] Processing CreateTablet for tablet 13d8cbcab5504f22a772e3f3f509ef7b (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:56:35.009654 15726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a37d408cd1479e93d53989431583f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.004992 15725 tablet_service.cc:1467] Processing CreateTablet for tablet a08aeabd9c134e689630f7828f0c4141 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:56:35.015497 15725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a08aeabd9c134e689630f7828f0c4141. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.015887 15724 tablet_service.cc:1467] Processing CreateTablet for tablet 355ac9e86ce94946bb77e763ce9c0731 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:56:35.017771 15723 tablet_service.cc:1467] Processing CreateTablet for tablet 13d8cbcab5504f22a772e3f3f509ef7b (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:56:35.020489 15575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 355ac9e86ce94946bb77e763ce9c0731. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.022485 15573 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7944602db44c8794b4bebaaf1f453a (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:56:35.024992 15723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d8cbcab5504f22a772e3f3f509ef7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.027788 15724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 355ac9e86ce94946bb77e763ce9c0731. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.035535 15576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a08aeabd9c134e689630f7828f0c4141. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.038237 15574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d8cbcab5504f22a772e3f3f509ef7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.039816 15573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7944602db44c8794b4bebaaf1f453a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.059003 15722 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7944602db44c8794b4bebaaf1f453a (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:56:35.059795 15572 tablet_service.cc:1467] Processing CreateTablet for tablet e0d2890cbd734733a7cb83d5e91d0d60 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:56:35.060272 15722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7944602db44c8794b4bebaaf1f453a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.061065 15572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0d2890cbd734733a7cb83d5e91d0d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.072499 15721 tablet_service.cc:1467] Processing CreateTablet for tablet e0d2890cbd734733a7cb83d5e91d0d60 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:56:35.073338 15571 tablet_service.cc:1467] Processing CreateTablet for tablet b38b65b2ea0944e2a1c04e31efb36270 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:56:35.073863 15721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0d2890cbd734733a7cb83d5e91d0d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.074640 15571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b38b65b2ea0944e2a1c04e31efb36270. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.108851 15720 tablet_service.cc:1467] Processing CreateTablet for tablet b38b65b2ea0944e2a1c04e31efb36270 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:56:35.129011 15720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b38b65b2ea0944e2a1c04e31efb36270. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.134713 15781 tablet_bootstrap.cc:492] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.135941 15782 tablet_bootstrap.cc:492] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.143529 15653 tablet_service.cc:1467] Processing CreateTablet for tablet 46abc6e120e3421e80afc6dd0d5d84c9 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:56:35.145959 15654 tablet_service.cc:1467] Processing CreateTablet for tablet 5984e78e6a014ec6951f87f52ca0c5e4 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:56:35.138525 15652 tablet_service.cc:1467] Processing CreateTablet for tablet 78a37d408cd1479e93d53989431583f3 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:56:35.148478 15651 tablet_service.cc:1467] Processing CreateTablet for tablet a08aeabd9c134e689630f7828f0c4141 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:56:35.162252 15653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46abc6e120e3421e80afc6dd0d5d84c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.165247 15650 tablet_service.cc:1467] Processing CreateTablet for tablet 355ac9e86ce94946bb77e763ce9c0731 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:56:35.168462 15646 tablet_service.cc:1467] Processing CreateTablet for tablet b38b65b2ea0944e2a1c04e31efb36270 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:56:35.168534 15647 tablet_service.cc:1467] Processing CreateTablet for tablet e0d2890cbd734733a7cb83d5e91d0d60 (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:56:35.172470 15648 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7944602db44c8794b4bebaaf1f453a (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:56:35.172330 15649 tablet_service.cc:1467] Processing CreateTablet for tablet 13d8cbcab5504f22a772e3f3f509ef7b (DEFAULT_TABLE table=test-workload [id=c8386c9dbf0046ae82cf4ef965d43cab]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:56:35.177114 15651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a08aeabd9c134e689630f7828f0c4141. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.178844 15646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b38b65b2ea0944e2a1c04e31efb36270. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.183125 15650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 355ac9e86ce94946bb77e763ce9c0731. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:56:35.206053 15533 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:56:35.203789 15654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5984e78e6a014ec6951f87f52ca0c5e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.214862 15648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7944602db44c8794b4bebaaf1f453a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.216519 15647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0d2890cbd734733a7cb83d5e91d0d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.234452 15649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13d8cbcab5504f22a772e3f3f509ef7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.254335 15652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a37d408cd1479e93d53989431583f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:56:35.280494 15782 tablet_bootstrap.cc:654] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.281212 15781 tablet_bootstrap.cc:654] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.295986 15782 tablet_bootstrap.cc:492] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.296852 15783 tablet_bootstrap.cc:492] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.303711 15782 ts_tablet_manager.cc:1397] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.169s user 0.014s sys 0.000s
I20250112 20:56:35.306035 15783 tablet_bootstrap.cc:654] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.306473 15782 raft_consensus.cc:357] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.307269 15782 raft_consensus.cc:383] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.307543 15782 raft_consensus.cc:738] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.309405 15782 consensus_queue.cc:260] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.312314 15781 tablet_bootstrap.cc:492] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.312856 15781 ts_tablet_manager.cc:1397] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.195s user 0.012s sys 0.016s
I20250112 20:56:35.316591 15781 raft_consensus.cc:357] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.317368 15781 raft_consensus.cc:383] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.317651 15781 raft_consensus.cc:738] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.318568 15783 tablet_bootstrap.cc:492] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.318315 15781 consensus_queue.cc:260] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.319054 15783 ts_tablet_manager.cc:1397] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.014s
I20250112 20:56:35.321413 15614 heartbeater.cc:502] Master 127.15.23.190:42983 was elected leader, sending a full tablet report...
I20250112 20:56:35.321732 15782 ts_tablet_manager.cc:1428] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
I20250112 20:56:35.321775 15783 raft_consensus.cc:357] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.322543 15783 raft_consensus.cc:383] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.322782 15782 tablet_bootstrap.cc:492] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.322849 15783 raft_consensus.cc:738] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.323588 15783 consensus_queue.cc:260] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.328878 15782 tablet_bootstrap.cc:654] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.329381 15763 heartbeater.cc:502] Master 127.15.23.190:42983 was elected leader, sending a full tablet report...
I20250112 20:56:35.331339 15783 ts_tablet_manager.cc:1428] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.012s user 0.004s sys 0.005s
I20250112 20:56:35.331930 15689 heartbeater.cc:502] Master 127.15.23.190:42983 was elected leader, sending a full tablet report...
I20250112 20:56:35.336494 15783 tablet_bootstrap.cc:492] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.342958 15783 tablet_bootstrap.cc:654] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.350526 15782 tablet_bootstrap.cc:492] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.350946 15782 ts_tablet_manager.cc:1397] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.028s user 0.018s sys 0.003s
I20250112 20:56:35.351763 15783 tablet_bootstrap.cc:492] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.352219 15783 ts_tablet_manager.cc:1397] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.005s
I20250112 20:56:35.353420 15782 raft_consensus.cc:357] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.354127 15782 raft_consensus.cc:383] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.354389 15782 raft_consensus.cc:738] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.354816 15783 raft_consensus.cc:357] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.354998 15782 consensus_queue.cc:260] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.355551 15783 raft_consensus.cc:383] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.355839 15783 raft_consensus.cc:738] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.356735 15783 consensus_queue.cc:260] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.357276 15782 ts_tablet_manager.cc:1428] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20250112 20:56:35.357205 15781 ts_tablet_manager.cc:1428] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.044s user 0.006s sys 0.002s
I20250112 20:56:35.358659 15782 tablet_bootstrap.cc:492] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.358829 15781 tablet_bootstrap.cc:492] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.359241 15783 ts_tablet_manager.cc:1428] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20250112 20:56:35.360076 15783 tablet_bootstrap.cc:492] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.365002 15782 tablet_bootstrap.cc:654] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.366204 15783 tablet_bootstrap.cc:654] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.375871 15782 tablet_bootstrap.cc:492] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.376293 15782 ts_tablet_manager.cc:1397] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.009s
I20250112 20:56:35.377265 15783 tablet_bootstrap.cc:492] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.377676 15783 ts_tablet_manager.cc:1397] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.003s
I20250112 20:56:35.378738 15782 raft_consensus.cc:357] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.379467 15782 raft_consensus.cc:383] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.379719 15782 raft_consensus.cc:738] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.380307 15782 consensus_queue.cc:260] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.380379 15783 raft_consensus.cc:357] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.381218 15783 raft_consensus.cc:383] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.381508 15783 raft_consensus.cc:738] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.382122 15783 consensus_queue.cc:260] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.382755 15782 ts_tablet_manager.cc:1428] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250112 20:56:35.383630 15782 tablet_bootstrap.cc:492] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.384682 15783 ts_tablet_manager.cc:1428] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20250112 20:56:35.385535 15783 tablet_bootstrap.cc:492] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.389381 15782 tablet_bootstrap.cc:654] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.391458 15783 tablet_bootstrap.cc:654] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.400487 15782 tablet_bootstrap.cc:492] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.400925 15782 ts_tablet_manager.cc:1397] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.013s
I20250112 20:56:35.403121 15783 tablet_bootstrap.cc:492] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.403577 15783 ts_tablet_manager.cc:1397] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.002s
I20250112 20:56:35.403429 15782 raft_consensus.cc:357] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.404183 15782 raft_consensus.cc:383] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.404481 15782 raft_consensus.cc:738] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.405136 15782 consensus_queue.cc:260] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.406308 15783 raft_consensus.cc:357] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.407024 15783 raft_consensus.cc:383] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.407284 15783 raft_consensus.cc:738] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.407624 15782 ts_tablet_manager.cc:1428] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:56:35.407898 15783 consensus_queue.cc:260] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.410420 15783 ts_tablet_manager.cc:1428] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20250112 20:56:35.411267 15783 tablet_bootstrap.cc:492] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.411312 15782 tablet_bootstrap.cc:492] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.417953 15782 tablet_bootstrap.cc:654] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.418198 15783 tablet_bootstrap.cc:654] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.420908 15781 tablet_bootstrap.cc:654] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
W20250112 20:56:35.425007 15690 tablet.cc:2367] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 20:56:35.427402 15787 raft_consensus.cc:491] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:35.427875 15787 raft_consensus.cc:513] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.430131 15783 tablet_bootstrap.cc:492] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.430543 15787 leader_election.cc:290] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:35.430541 15783 ts_tablet_manager.cc:1397] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.004s
I20250112 20:56:35.434034 15783 raft_consensus.cc:357] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.434798 15783 raft_consensus.cc:383] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.435098 15783 raft_consensus.cc:738] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.435897 15783 consensus_queue.cc:260] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.438685 15783 ts_tablet_manager.cc:1428] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20250112 20:56:35.439764 15783 tablet_bootstrap.cc:492] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.443914 15781 tablet_bootstrap.cc:492] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.444460 15781 ts_tablet_manager.cc:1397] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.086s user 0.009s sys 0.009s
I20250112 20:56:35.455210 15783 tablet_bootstrap.cc:654] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.460151 15781 raft_consensus.cc:357] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.461989 15781 raft_consensus.cc:383] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.462329 15783 tablet_bootstrap.cc:492] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.462291 15781 raft_consensus.cc:738] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.462829 15783 ts_tablet_manager.cc:1397] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.004s
I20250112 20:56:35.463150 15781 consensus_queue.cc:260] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.465651 15783 raft_consensus.cc:357] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.466394 15783 raft_consensus.cc:383] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.466686 15783 raft_consensus.cc:738] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.467350 15783 consensus_queue.cc:260] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.470227 15782 tablet_bootstrap.cc:492] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.470414 15783 ts_tablet_manager.cc:1428] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20250112 20:56:35.470839 15782 ts_tablet_manager.cc:1397] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.060s user 0.009s sys 0.034s
I20250112 20:56:35.471475 15783 tablet_bootstrap.cc:492] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.474443 15782 raft_consensus.cc:357] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.475570 15789 raft_consensus.cc:491] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:35.476830 15782 raft_consensus.cc:383] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.477268 15782 raft_consensus.cc:738] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.476922 15789 raft_consensus.cc:513] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.478071 15782 consensus_queue.cc:260] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.478915 15783 tablet_bootstrap.cc:654] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.479471 15789 leader_election.cc:290] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.480863 15782 ts_tablet_manager.cc:1428] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20250112 20:56:35.481941 15782 tablet_bootstrap.cc:492] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.489567 15782 tablet_bootstrap.cc:654] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.492969 15783 tablet_bootstrap.cc:492] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.497031 15783 ts_tablet_manager.cc:1397] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.006s
I20250112 20:56:35.497262 15789 raft_consensus.cc:491] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:35.497704 15789 raft_consensus.cc:513] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.499593 15789 leader_election.cc:290] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.500097 15783 raft_consensus.cc:357] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.500885 15783 raft_consensus.cc:383] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.501204 15783 raft_consensus.cc:738] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.504206 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38b65b2ea0944e2a1c04e31efb36270" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:35.503073 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:35.505528 15588 raft_consensus.cc:2463] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 0.
W20250112 20:56:35.507068 15627 leader_election.cc:343] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.508332 15627 leader_election.cc:304] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
I20250112 20:56:35.503053 15783 consensus_queue.cc:260] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.511363 15783 ts_tablet_manager.cc:1428] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20250112 20:56:35.512539 15789 raft_consensus.cc:2798] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:35.512270 15783 tablet_bootstrap.cc:492] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.513077 15789 raft_consensus.cc:491] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:56:35.513880 15781 ts_tablet_manager.cc:1428] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.069s user 0.006s sys 0.001s
I20250112 20:56:35.514333 15789 raft_consensus.cc:3054] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.517391 15782 tablet_bootstrap.cc:492] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.517798 15782 ts_tablet_manager.cc:1397] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.036s user 0.011s sys 0.001s
I20250112 20:56:35.520803 15781 tablet_bootstrap.cc:492] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.526870 15738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:35.523964 15782 raft_consensus.cc:357] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.528522 15738 raft_consensus.cc:2463] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 0.
I20250112 20:56:35.528779 15782 raft_consensus.cc:383] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.531459 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:35.533355 15664 raft_consensus.cc:2463] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 0.
I20250112 20:56:35.534987 15781 tablet_bootstrap.cc:654] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.535086 15789 raft_consensus.cc:513] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.530941 15782 raft_consensus.cc:738] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.538622 15789 leader_election.cc:290] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.539088 15789 raft_consensus.cc:491] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:35.531977 15555 leader_election.cc:304] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:35.548862 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38b65b2ea0944e2a1c04e31efb36270" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:35.542162 15782 consensus_queue.cc:260] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.571096 15738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:35.564373 15787 raft_consensus.cc:2798] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:35.592217 15787 raft_consensus.cc:491] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20250112 20:56:35.592386 15630 leader_election.cc:343] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.592615 15787 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.575881 15788 raft_consensus.cc:491] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250112 20:56:35.593457 15630 leader_election.cc:343] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.593240 15788 raft_consensus.cc:513] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.594005 15630 leader_election.cc:304] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f; no voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.595839 15788 leader_election.cc:290] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:35.572568 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38b65b2ea0944e2a1c04e31efb36270" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:35.595568 15807 raft_consensus.cc:2743] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:56:35.597290 15588 raft_consensus.cc:3054] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.543783 15789 raft_consensus.cc:513] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.556298 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38b65b2ea0944e2a1c04e31efb36270" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:35.603104 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
W20250112 20:56:35.605377 15630 leader_election.cc:343] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:56:35.606211 15627 leader_election.cc:343] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.587177 15783 tablet_bootstrap.cc:654] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.608180 15789 leader_election.cc:290] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.612361 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:35.614609 15782 ts_tablet_manager.cc:1428] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.097s user 0.029s sys 0.017s
I20250112 20:56:35.616552 15588 raft_consensus.cc:2463] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 1.
W20250112 20:56:35.626492 15630 leader_election.cc:343] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.630467 15782 tablet_bootstrap.cc:492] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.624163 15787 raft_consensus.cc:513] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.630628 15627 leader_election.cc:304] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters: f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.631661 15789 raft_consensus.cc:2798] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:35.632128 15789 raft_consensus.cc:695] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:35.632364 15787 leader_election.cc:290] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Requested vote from peers f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:35.633519 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:35.634248 15737 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.636744 15782 tablet_bootstrap.cc:654] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.638118 15789 consensus_queue.cc:237] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.616020 15781 tablet_bootstrap.cc:492] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.642282 15630 leader_election.cc:304] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f; no voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.643759 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:35.644572 15664 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.644762 15781 ts_tablet_manager.cc:1397] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.124s user 0.011s sys 0.007s
I20250112 20:56:35.645633 15807 raft_consensus.cc:2743] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:56:35.650435 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:35.651055 15588 raft_consensus.cc:2463] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 0.
I20250112 20:56:35.656692 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:35.665484 15663 raft_consensus.cc:2463] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 0.
I20250112 20:56:35.669116 15737 raft_consensus.cc:2463] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 1.
I20250112 20:56:35.668998 15781 raft_consensus.cc:357] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.669704 15781 raft_consensus.cc:383] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.669814 15787 raft_consensus.cc:491] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:35.669996 15781 raft_consensus.cc:738] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.670221 15787 raft_consensus.cc:513] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.670624 15781 consensus_queue.cc:260] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.672096 15787 leader_election.cc:290] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.672227 15555 leader_election.cc:304] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:35.667549 15664 raft_consensus.cc:2463] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 1.
I20250112 20:56:35.673162 15787 raft_consensus.cc:2798] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:35.673588 15787 raft_consensus.cc:695] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:35.674439 15787 consensus_queue.cc:237] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:35.675617 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
W20250112 20:56:35.676981 15555 leader_election.cc:343] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.655755 15701 leader_election.cc:304] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:35.680176 15788 raft_consensus.cc:2798] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:35.680547 15788 raft_consensus.cc:491] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:56:35.680817 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:35.681044 15788 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.679550 15783 tablet_bootstrap.cc:492] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.682271 15783 ts_tablet_manager.cc:1397] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.170s user 0.018s sys 0.031s
I20250112 20:56:35.682006 15664 raft_consensus.cc:2463] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 0.
I20250112 20:56:35.691514 15552 leader_election.cc:304] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters: f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.693229 15782 tablet_bootstrap.cc:492] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.693783 15787 raft_consensus.cc:2798] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:35.693919 15782 ts_tablet_manager.cc:1397] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.064s user 0.002s sys 0.021s
I20250112 20:56:35.694156 15787 raft_consensus.cc:491] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:56:35.694942 15787 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.696905 15782 raft_consensus.cc:357] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.697531 15782 raft_consensus.cc:383] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.697813 15782 raft_consensus.cc:738] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.699301 15782 consensus_queue.cc:260] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.700114 15781 ts_tablet_manager.cc:1428] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.055s user 0.008s sys 0.000s
I20250112 20:56:35.700546 15486 catalog_manager.cc:5526] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:35.701655 15782 ts_tablet_manager.cc:1428] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20250112 20:56:35.703210 15782 tablet_bootstrap.cc:492] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.704877 15787 raft_consensus.cc:513] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.705569 15781 tablet_bootstrap.cc:492] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.704514 15783 raft_consensus.cc:357] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.707049 15783 raft_consensus.cc:383] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.707345 15783 raft_consensus.cc:738] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.707816 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:35.707878 15783 consensus_queue.cc:260] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.708386 15664 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.708563 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
W20250112 20:56:35.709969 15555 leader_election.cc:343] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.702754 15483 catalog_manager.cc:5526] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 1 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:35.706655 15787 leader_election.cc:290] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Requested vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.717573 15664 raft_consensus.cc:2463] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 1.
I20250112 20:56:35.712126 15788 raft_consensus.cc:513] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.718681 15552 leader_election.cc:304] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters: f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.721930 15783 ts_tablet_manager.cc:1428] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.039s user 0.011s sys 0.007s
I20250112 20:56:35.721125 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:35.721745 15787 raft_consensus.cc:2798] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:35.722787 15783 tablet_bootstrap.cc:492] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f: Bootstrap starting.
I20250112 20:56:35.722697 15588 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.722972 15787 raft_consensus.cc:695] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:35.724009 15787 consensus_queue.cc:237] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.721452 15788 leader_election.cc:290] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:35.726220 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:35.727202 15664 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.728685 15789 raft_consensus.cc:491] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:35.729182 15588 raft_consensus.cc:2463] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 1.
I20250112 20:56:35.729132 15789 raft_consensus.cc:513] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.730571 15701 leader_election.cc:304] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:35.731602 15789 leader_election.cc:290] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.732735 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7944602db44c8794b4bebaaf1f453a" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:35.733400 15788 raft_consensus.cc:2798] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:35.734412 15783 tablet_bootstrap.cc:654] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f: Neither blocks nor log segments found. Creating new log.
W20250112 20:56:35.734294 15630 leader_election.cc:343] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.735728 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7944602db44c8794b4bebaaf1f453a" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:35.737180 15588 raft_consensus.cc:2463] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 0.
I20250112 20:56:35.740821 15788 raft_consensus.cc:695] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 LEADER]: Becoming Leader. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:35.742799 15627 leader_election.cc:304] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters: f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.743829 15789 raft_consensus.cc:2798] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:35.745167 15789 raft_consensus.cc:491] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:56:35.745558 15789 raft_consensus.cc:3054] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.749071 15664 raft_consensus.cc:2463] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 1.
I20250112 20:56:35.756255 15783 tablet_bootstrap.cc:492] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f: No bootstrap required, opened a new log
I20250112 20:56:35.747748 15788 consensus_queue.cc:237] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.756832 15783 ts_tablet_manager.cc:1397] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f: Time spent bootstrapping tablet: real 0.034s user 0.016s sys 0.001s
I20250112 20:56:35.765995 15782 tablet_bootstrap.cc:654] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.766213 15789 raft_consensus.cc:513] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.775590 15781 tablet_bootstrap.cc:654] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.785084 15789 leader_election.cc:290] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.788084 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7944602db44c8794b4bebaaf1f453a" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:35.788791 15783 raft_consensus.cc:357] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.789278 15783 raft_consensus.cc:383] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.789479 15783 raft_consensus.cc:738] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Initialized, Role: FOLLOWER
W20250112 20:56:35.789661 15630 leader_election.cc:343] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.789948 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7944602db44c8794b4bebaaf1f453a" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:35.790041 15783 consensus_queue.cc:260] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.790524 15588 raft_consensus.cc:3054] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.792083 15783 ts_tablet_manager.cc:1428] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f: Time spent starting tablet: real 0.035s user 0.003s sys 0.000s
I20250112 20:56:35.805727 15588 raft_consensus.cc:2463] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 1.
I20250112 20:56:35.806255 15782 tablet_bootstrap.cc:492] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.807090 15782 ts_tablet_manager.cc:1397] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.104s user 0.015s sys 0.006s
I20250112 20:56:35.805732 15781 tablet_bootstrap.cc:492] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.808172 15627 leader_election.cc:304] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters: f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.809608 15781 ts_tablet_manager.cc:1397] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.104s user 0.011s sys 0.006s
I20250112 20:56:35.810618 15807 raft_consensus.cc:2798] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:35.811659 15807 raft_consensus.cc:695] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:35.812041 15782 raft_consensus.cc:357] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.813452 15782 raft_consensus.cc:383] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.813661 15782 raft_consensus.cc:738] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.814154 15782 consensus_queue.cc:260] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.814829 15787 raft_consensus.cc:491] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:35.823731 15787 raft_consensus.cc:513] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.812646 15807 consensus_queue.cc:237] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.835191 15787 leader_election.cc:290] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.836570 15782 ts_tablet_manager.cc:1428] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.029s user 0.005s sys 0.001s
I20250112 20:56:35.836872 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:35.837401 15782 tablet_bootstrap.cc:492] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1: Bootstrap starting.
I20250112 20:56:35.837154 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:35.837723 15664 raft_consensus.cc:2463] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 0.
I20250112 20:56:35.839355 15781 raft_consensus.cc:357] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.838811 15552 leader_election.cc:304] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
W20250112 20:56:35.841333 15555 leader_election.cc:343] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.841943 15787 raft_consensus.cc:2798] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:35.842375 15787 raft_consensus.cc:491] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:56:35.842728 15787 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.847658 15809 raft_consensus.cc:491] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:35.848980 15787 raft_consensus.cc:513] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.853205 15809 raft_consensus.cc:513] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.844571 15485 catalog_manager.cc:5526] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: HEALTHY } } }
I20250112 20:56:35.855213 15809 leader_election.cc:290] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.855996 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:35.856101 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:35.856395 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:35.856673 15664 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.857131 15663 raft_consensus.cc:2463] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 0.
W20250112 20:56:35.857617 15555 leader_election.cc:343] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.858244 15552 leader_election.cc:304] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters: f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.860102 15787 leader_election.cc:290] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Requested vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.860692 15787 raft_consensus.cc:2798] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:35.861063 15787 raft_consensus.cc:491] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:56:35.861420 15787 raft_consensus.cc:3054] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.850049 15782 tablet_bootstrap.cc:654] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.862318 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
W20250112 20:56:35.863554 15555 leader_election.cc:343] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.852560 15486 catalog_manager.cc:5526] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 1 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:35.868434 15782 tablet_bootstrap.cc:492] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1: No bootstrap required, opened a new log
I20250112 20:56:35.868986 15782 ts_tablet_manager.cc:1397] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.005s
I20250112 20:56:35.841241 15781 raft_consensus.cc:383] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.870546 15781 raft_consensus.cc:738] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.871352 15781 consensus_queue.cc:260] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.871757 15782 raft_consensus.cc:357] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.872404 15782 raft_consensus.cc:383] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.872733 15782 raft_consensus.cc:738] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.872856 15787 raft_consensus.cc:513] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.873365 15782 consensus_queue.cc:260] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.874428 15781 ts_tablet_manager.cc:1428] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.064s user 0.006s sys 0.002s
I20250112 20:56:35.876781 15781 tablet_bootstrap.cc:492] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.877089 15782 ts_tablet_manager.cc:1428] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250112 20:56:35.880694 15664 raft_consensus.cc:2463] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 1.
I20250112 20:56:35.882125 15552 leader_election.cc:304] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters: f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.883270 15809 raft_consensus.cc:2798] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:35.874675 15787 leader_election.cc:290] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Requested vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:35.875551 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:35.888377 15663 raft_consensus.cc:3054] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:35.890209 15483 catalog_manager.cc:5526] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:35.875733 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
W20250112 20:56:35.896441 15555 leader_election.cc:343] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:56:35.901041 15809 raft_consensus.cc:695] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:35.902369 15663 raft_consensus.cc:2463] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 1.
I20250112 20:56:35.903410 15552 leader_election.cc:304] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters: f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:35.905133 15781 tablet_bootstrap.cc:654] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.905176 15828 raft_consensus.cc:2798] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:35.907887 15828 raft_consensus.cc:695] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:35.905726 15809 consensus_queue.cc:237] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.908942 15828 consensus_queue.cc:237] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.917517 15781 tablet_bootstrap.cc:492] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.918040 15781 ts_tablet_manager.cc:1397] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.041s user 0.011s sys 0.006s
I20250112 20:56:35.921525 15781 raft_consensus.cc:357] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.922484 15781 raft_consensus.cc:383] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.922899 15781 raft_consensus.cc:738] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.923718 15781 consensus_queue.cc:260] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.927042 15781 ts_tablet_manager.cc:1428] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20250112 20:56:35.927884 15781 tablet_bootstrap.cc:492] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.932716 15486 catalog_manager.cc:5526] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:35.934597 15781 tablet_bootstrap.cc:654] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.934593 15486 catalog_manager.cc:5526] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:35.943524 15781 tablet_bootstrap.cc:492] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.943941 15781 ts_tablet_manager.cc:1397] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.003s
I20250112 20:56:35.945786 15781 raft_consensus.cc:357] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.946508 15781 raft_consensus.cc:383] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.946795 15781 raft_consensus.cc:738] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.947276 15781 consensus_queue.cc:260] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.949187 15781 ts_tablet_manager.cc:1428] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20250112 20:56:35.950074 15781 tablet_bootstrap.cc:492] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.955816 15781 tablet_bootstrap.cc:654] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.960805 15781 tablet_bootstrap.cc:492] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.961125 15781 ts_tablet_manager.cc:1397] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20250112 20:56:35.963425 15781 raft_consensus.cc:357] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.964148 15781 raft_consensus.cc:383] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.964442 15781 raft_consensus.cc:738] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.965713 15781 consensus_queue.cc:260] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.967515 15781 ts_tablet_manager.cc:1428] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:56:35.968344 15781 tablet_bootstrap.cc:492] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.974859 15781 tablet_bootstrap.cc:654] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.978713 15781 tablet_bootstrap.cc:492] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.979012 15781 ts_tablet_manager.cc:1397] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20250112 20:56:35.980597 15781 raft_consensus.cc:357] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.981109 15781 raft_consensus.cc:383] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.981302 15781 raft_consensus.cc:738] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.981725 15781 consensus_queue.cc:260] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.983573 15781 ts_tablet_manager.cc:1428] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20250112 20:56:35.984375 15781 tablet_bootstrap.cc:492] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5: Bootstrap starting.
I20250112 20:56:35.989061 15781 tablet_bootstrap.cc:654] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5: Neither blocks nor log segments found. Creating new log.
I20250112 20:56:35.993125 15781 tablet_bootstrap.cc:492] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5: No bootstrap required, opened a new log
I20250112 20:56:35.993492 15781 ts_tablet_manager.cc:1397] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.001s
I20250112 20:56:35.995110 15781 raft_consensus.cc:357] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.995611 15781 raft_consensus.cc:383] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:56:35.995807 15781 raft_consensus.cc:738] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Initialized, Role: FOLLOWER
I20250112 20:56:35.996264 15781 consensus_queue.cc:260] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:35.998065 15781 ts_tablet_manager.cc:1428] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20250112 20:56:36.041869 15807 raft_consensus.cc:491] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:36.042379 15807 raft_consensus.cc:513] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.044623 15807 leader_election.cc:290] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:36.045161 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:36.045785 15588 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 0.
I20250112 20:56:36.045994 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:36.046571 15737 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 0.
I20250112 20:56:36.046725 15627 leader_election.cc:304] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
I20250112 20:56:36.047457 15807 raft_consensus.cc:2798] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:36.047782 15807 raft_consensus.cc:491] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:56:36.048023 15807 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.052490 15807 raft_consensus.cc:513] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.054071 15807 leader_election.cc:290] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:36.054786 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:36.055092 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:36.055369 15588 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.055568 15737 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.060039 15737 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 1.
I20250112 20:56:36.060837 15630 leader_election.cc:304] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:36.061219 15588 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 1.
I20250112 20:56:36.061404 15807 raft_consensus.cc:2798] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:36.061841 15807 raft_consensus.cc:695] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:36.062443 15807 consensus_queue.cc:237] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.069437 15486 catalog_manager.cc:5526] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 1 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:36.083802 15828 raft_consensus.cc:491] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:36.084239 15828 raft_consensus.cc:513] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.085803 15828 leader_election.cc:290] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:36.086541 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:36.086709 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" is_pre_election: true
I20250112 20:56:36.087196 15663 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 0.
I20250112 20:56:36.087252 15737 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 0.
I20250112 20:56:36.088303 15552 leader_election.cc:304] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
I20250112 20:56:36.089018 15828 raft_consensus.cc:2798] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:56:36.089335 15828 raft_consensus.cc:491] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:56:36.089648 15828 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.094877 15828 raft_consensus.cc:513] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.096246 15828 leader_election.cc:290] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Requested vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:36.097062 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:36.097257 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:36.097501 15663 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.097826 15737 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.101819 15663 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 1.
I20250112 20:56:36.102679 15552 leader_election.cc:304] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
I20250112 20:56:36.103278 15828 raft_consensus.cc:2798] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:56:36.103559 15737 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 1.
I20250112 20:56:36.103629 15828 raft_consensus.cc:695] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:36.104509 15828 consensus_queue.cc:237] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.112787 15483 catalog_manager.cc:5526] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:36.123600 15828 consensus_queue.cc:1035] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.138216 15787 consensus_queue.cc:1035] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.152837 15736 raft_consensus.cc:3054] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.160612 15807 consensus_queue.cc:1035] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.170895 15807 consensus_queue.cc:1035] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.194875 15788 raft_consensus.cc:491] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:36.195302 15788 raft_consensus.cc:513] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.196761 15788 leader_election.cc:290] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:36.197547 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:36.197875 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:36.198617 15663 raft_consensus.cc:2388] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f0e345cb8d6f447bb560e3e0b204c2c5 in current term 1: Already voted for candidate 546138398ea14db0bcfe85636a6eebe1 in this term.
I20250112 20:56:36.199414 15701 leader_election.cc:304] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0e345cb8d6f447bb560e3e0b204c2c5; no voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1
I20250112 20:56:36.199999 15788 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.205406 15788 raft_consensus.cc:2743] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:56:36.208271 15788 consensus_queue.cc:1035] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.214249 15788 raft_consensus.cc:491] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:36.214648 15788 raft_consensus.cc:513] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.216338 15788 leader_election.cc:290] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:36.217773 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:36.219264 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:36.219969 15663 raft_consensus.cc:2388] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f0e345cb8d6f447bb560e3e0b204c2c5 in current term 1: Already voted for candidate 546138398ea14db0bcfe85636a6eebe1 in this term.
I20250112 20:56:36.221274 15701 leader_election.cc:304] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0e345cb8d6f447bb560e3e0b204c2c5; no voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1
I20250112 20:56:36.222308 15788 raft_consensus.cc:3054] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.226675 15788 raft_consensus.cc:2743] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:56:36.231032 15817 consensus_queue.cc:1035] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.246932 15787 consensus_queue.cc:1035] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.258256 15828 consensus_queue.cc:1035] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:56:36.301908 15787 consensus_queue.cc:1035] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.307005 15817 raft_consensus.cc:491] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:36.307426 15817 raft_consensus.cc:513] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.309768 15817 leader_election.cc:290] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:36.312345 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7944602db44c8794b4bebaaf1f453a" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:36.313738 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7944602db44c8794b4bebaaf1f453a" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:36.314503 15588 raft_consensus.cc:2388] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f0e345cb8d6f447bb560e3e0b204c2c5 in current term 1: Already voted for candidate 3105649bf63c47b6b4c11216164d816f in this term.
I20250112 20:56:36.315116 15809 consensus_queue.cc:1035] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.319609 15701 leader_election.cc:304] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0e345cb8d6f447bb560e3e0b204c2c5; no voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1
I20250112 20:56:36.321239 15817 raft_consensus.cc:3054] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.327970 15817 raft_consensus.cc:2743] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:56:36.335804 15817 raft_consensus.cc:491] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:56:36.336126 15817 raft_consensus.cc:513] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.337540 15817 leader_election.cc:290] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:36.338224 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "546138398ea14db0bcfe85636a6eebe1" is_pre_election: true
I20250112 20:56:36.338403 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3105649bf63c47b6b4c11216164d816f" is_pre_election: true
I20250112 20:56:36.339150 15663 raft_consensus.cc:2388] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f0e345cb8d6f447bb560e3e0b204c2c5 in current term 1: Already voted for candidate 546138398ea14db0bcfe85636a6eebe1 in this term.
I20250112 20:56:36.339952 15701 leader_election.cc:304] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0e345cb8d6f447bb560e3e0b204c2c5; no voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1
I20250112 20:56:36.341044 15817 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:56:36.346133 15817 raft_consensus.cc:2743] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:56:36.364341 15850 consensus_queue.cc:1035] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.365082 15809 consensus_queue.cc:1035] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.390247 15807 consensus_queue.cc:1035] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.411695 15807 consensus_queue.cc:1035] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.436718 15663 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "78a37d408cd1479e93d53989431583f3"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
mode: GRACEFUL
new_leader_uuid: "546138398ea14db0bcfe85636a6eebe1"
from {username='slave'} at 127.0.0.1:44684
I20250112 20:56:36.437354 15663 raft_consensus.cc:604] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Received request to transfer leadership to TS 546138398ea14db0bcfe85636a6eebe1
I20250112 20:56:36.453701 15589 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
mode: GRACEFUL
new_leader_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:45924
I20250112 20:56:36.454213 15589 raft_consensus.cc:604] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Received request to transfer leadership to TS 3105649bf63c47b6b4c11216164d816f
I20250112 20:56:36.459154 15589 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
mode: GRACEFUL
new_leader_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:45924
I20250112 20:56:36.459581 15589 raft_consensus.cc:604] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Received request to transfer leadership to TS 3105649bf63c47b6b4c11216164d816f
I20250112 20:56:36.473578 15737 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60"
dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
mode: GRACEFUL
new_leader_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:33424
I20250112 20:56:36.474184 15737 raft_consensus.cc:604] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 LEADER]: Received request to transfer leadership to TS 3105649bf63c47b6b4c11216164d816f
I20250112 20:56:36.478978 15663 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b38b65b2ea0944e2a1c04e31efb36270"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
mode: GRACEFUL
new_leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:44684
I20250112 20:56:36.479528 15663 raft_consensus.cc:604] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Received request to transfer leadership to TS f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:36.593077 15787 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.595120 15807 consensus_queue.cc:1035] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.609323 15844 raft_consensus.cc:988] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f: : Instructing follower 546138398ea14db0bcfe85636a6eebe1 to start an election
I20250112 20:56:36.609768 15844 raft_consensus.cc:1076] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Signalling peer 546138398ea14db0bcfe85636a6eebe1 to start an election
I20250112 20:56:36.613548 15844 consensus_queue.cc:1035] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.616241 15588 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "78a37d408cd1479e93d53989431583f3"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
from {username='slave'} at 127.0.0.1:45906
I20250112 20:56:36.616855 15588 raft_consensus.cc:491] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:36.617204 15588 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.623340 15787 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:56:36.631435 15588 raft_consensus.cc:513] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.633118 15589 raft_consensus.cc:1235] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Rejecting Update request from peer 3105649bf63c47b6b4c11216164d816f for earlier term 1. Current term is 2. Ops: []
I20250112 20:56:36.635852 15807 raft_consensus.cc:988] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f: : Instructing follower f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:36.636281 15807 raft_consensus.cc:1076] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Signalling peer f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:36.638463 15737 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b38b65b2ea0944e2a1c04e31efb36270"
dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:33414
I20250112 20:56:36.639032 15737 raft_consensus.cc:491] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:36.639400 15737 raft_consensus.cc:3054] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.646298 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:36.646898 15664 raft_consensus.cc:3049] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:56:36.647220 15664 raft_consensus.cc:738] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:36.647977 15664 consensus_queue.cc:260] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.649252 15664 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.654139 15588 leader_election.cc:290] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 2 election: Requested vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:36.655659 15664 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 2.
I20250112 20:56:36.656792 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:36.657384 15736 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.657469 15552 leader_election.cc:304] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
I20250112 20:56:36.658221 15787 raft_consensus.cc:2798] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:56:36.658648 15787 raft_consensus.cc:695] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:36.659385 15787 consensus_queue.cc:237] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.660269 15737 raft_consensus.cc:513] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.663465 15737 leader_election.cc:290] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 2 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:36.663622 15736 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 2.
I20250112 20:56:36.663867 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38b65b2ea0944e2a1c04e31efb36270" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:36.664789 15588 raft_consensus.cc:3054] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.664834 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b38b65b2ea0944e2a1c04e31efb36270" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:36.665563 15664 raft_consensus.cc:3049] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:56:36.665930 15664 raft_consensus.cc:738] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:36.666736 15664 consensus_queue.cc:260] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.668061 15664 raft_consensus.cc:3054] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.668015 15483 catalog_manager.cc:5526] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 1 to 2, leader changed from 3105649bf63c47b6b4c11216164d816f (127.15.23.130) to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:36.671581 15828 raft_consensus.cc:988] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1: : Instructing follower 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:36.672015 15828 raft_consensus.cc:1076] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Signalling peer 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:36.674938 15663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:44670
I20250112 20:56:36.676256 15663 raft_consensus.cc:491] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:36.676651 15663 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.679867 15588 raft_consensus.cc:2463] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 2.
I20250112 20:56:36.675166 15664 raft_consensus.cc:2463] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 2.
I20250112 20:56:36.680968 15701 leader_election.cc:304] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:36.682180 15817 raft_consensus.cc:2798] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:56:36.682610 15817 raft_consensus.cc:695] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 LEADER]: Becoming Leader. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:36.683403 15817 consensus_queue.cc:237] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.685182 15663 raft_consensus.cc:513] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.688735 15663 leader_election.cc:290] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:36.689152 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:36.689723 15589 raft_consensus.cc:3049] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:56:36.690019 15589 raft_consensus.cc:738] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:36.690634 15589 consensus_queue.cc:260] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.691746 15589 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.691928 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:36.692623 15736 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.693215 15484 catalog_manager.cc:5526] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 reported cstate change: term changed from 1 to 2, leader changed from 3105649bf63c47b6b4c11216164d816f (127.15.23.130) to f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131). New cstate: current_term: 2 leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: HEALTHY } } }
I20250112 20:56:36.698750 15736 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:36.699456 15589 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:36.699918 15630 leader_election.cc:304] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:36.700781 15807 raft_consensus.cc:2798] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:56:36.701174 15807 raft_consensus.cc:695] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:36.702224 15807 consensus_queue.cc:237] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.715637 15486 catalog_manager.cc:5526] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 1 to 2, leader changed from 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129) to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 2 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:36.832163 15870 raft_consensus.cc:988] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5: : Instructing follower 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:36.832798 15847 raft_consensus.cc:1076] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 LEADER]: Signalling peer 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:36.834143 15663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:44676
I20250112 20:56:36.834582 15663 raft_consensus.cc:491] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:36.834857 15663 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.839004 15663 raft_consensus.cc:513] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.840444 15663 leader_election.cc:290] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:36.841235 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:36.841399 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:36.841769 15589 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.841960 15736 raft_consensus.cc:3049] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:56:36.842265 15736 raft_consensus.cc:738] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:36.842876 15736 consensus_queue.cc:260] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.843744 15736 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.848346 15589 raft_consensus.cc:2463] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:36.849224 15736 raft_consensus.cc:2463] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:36.849431 15627 leader_election.cc:304] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
I20250112 20:56:36.850214 15807 raft_consensus.cc:2798] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:56:36.850637 15807 raft_consensus.cc:695] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:36.851483 15807 consensus_queue.cc:237] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.857584 15486 catalog_manager.cc:5526] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 1 to 2, leader changed from f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131) to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 2 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:36.976958 15828 raft_consensus.cc:988] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1: : Instructing follower 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:36.977351 15850 raft_consensus.cc:1076] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Signalling peer 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:36.978567 15663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:44670
I20250112 20:56:36.978981 15663 raft_consensus.cc:491] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:36.979212 15663 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.983197 15663 raft_consensus.cc:513] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.984503 15663 leader_election.cc:290] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:36.985199 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:36.985505 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:36.985723 15589 raft_consensus.cc:3049] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:56:36.986021 15589 raft_consensus.cc:738] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:36.986027 15736 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.986726 15589 consensus_queue.cc:260] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.987607 15589 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:36.990101 15736 raft_consensus.cc:2463] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:36.990999 15630 leader_election.cc:304] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:36.991767 15807 raft_consensus.cc:2798] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:56:36.991942 15589 raft_consensus.cc:2463] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:36.992224 15807 raft_consensus.cc:695] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:36.993052 15807 consensus_queue.cc:237] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:36.998242 15486 catalog_manager.cc:5526] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 1 to 2, leader changed from 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129) to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 2 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:37.105319 15663 raft_consensus.cc:1270] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Refusing update from remote peer 546138398ea14db0bcfe85636a6eebe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.106287 15850 consensus_queue.cc:1035] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.113337 15736 raft_consensus.cc:1270] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Refusing update from remote peer 546138398ea14db0bcfe85636a6eebe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.114751 15871 consensus_queue.cc:1035] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.129813 15589 raft_consensus.cc:1270] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.130831 15807 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.137054 15736 raft_consensus.cc:1270] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.138552 15844 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.149515 15663 raft_consensus.cc:1270] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.150511 15870 consensus_queue.cc:1035] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.156777 15589 raft_consensus.cc:1270] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.161257 15817 consensus_queue.cc:1035] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.277020 15736 raft_consensus.cc:1270] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.278086 15844 consensus_queue.cc:1035] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.285811 15589 raft_consensus.cc:1270] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.287281 15789 consensus_queue.cc:1035] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.374532 15589 raft_consensus.cc:1270] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.375607 15789 consensus_queue.cc:1035] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:37.383421 15737 raft_consensus.cc:1270] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:37.384997 15807 consensus_queue.cc:1035] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:39.547351 15454 auto_leader_rebalancer-test.cc:269] The leader distribution is
I20250112 20:56:39.548074 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:39.566606 15664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
mode: GRACEFUL
new_leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:44694
I20250112 20:56:39.567106 15664 raft_consensus.cc:604] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Received request to transfer leadership to TS f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:39.577878 15589 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
mode: GRACEFUL
new_leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:45938
I20250112 20:56:39.578315 15589 raft_consensus.cc:604] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Received request to transfer leadership to TS f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:39.579162 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:56:39.579533 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:39.668537 15885 raft_consensus.cc:988] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1: : Instructing follower f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:39.668999 15871 raft_consensus.cc:1076] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Signalling peer f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:39.670192 15737 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141"
dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:33410
I20250112 20:56:39.670612 15737 raft_consensus.cc:491] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:39.670876 15737 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:39.675252 15737 raft_consensus.cc:513] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:39.676561 15737 leader_election.cc:290] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 2 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:39.677294 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:39.677585 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:39.677909 15589 raft_consensus.cc:3049] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:56:39.678097 15664 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:39.678242 15589 raft_consensus.cc:738] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:39.679013 15589 consensus_queue.cc:260] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:39.680151 15589 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:39.684468 15664 raft_consensus.cc:2463] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 2.
I20250112 20:56:39.685572 15701 leader_election.cc:304] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:39.686291 15879 raft_consensus.cc:2798] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:56:39.686527 15589 raft_consensus.cc:2463] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 2.
I20250112 20:56:39.687366 15879 raft_consensus.cc:695] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 LEADER]: Becoming Leader. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:39.688115 15879 consensus_queue.cc:237] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:39.696491 15486 catalog_manager.cc:5526] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 reported cstate change: term changed from 1 to 2, leader changed from 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129) to f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131). New cstate: current_term: 2 leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: HEALTHY } } }
I20250112 20:56:39.724258 15881 raft_consensus.cc:988] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: : Instructing follower f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:39.724674 15880 raft_consensus.cc:1076] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Signalling peer f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:39.725883 15737 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731"
dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:33414
I20250112 20:56:39.726281 15737 raft_consensus.cc:491] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:39.726498 15737 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:39.730443 15737 raft_consensus.cc:513] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:39.731916 15737 leader_election.cc:290] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 3 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:39.732582 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:39.732980 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:39.733188 15589 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:39.733490 15664 raft_consensus.cc:3049] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:56:39.733752 15664 raft_consensus.cc:738] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:39.734409 15664 consensus_queue.cc:260] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:39.735497 15664 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:39.738651 15589 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 3.
I20250112 20:56:39.739502 15701 leader_election.cc:304] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:39.740341 15879 raft_consensus.cc:2798] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:56:39.740684 15879 raft_consensus.cc:695] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 LEADER]: Becoming Leader. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:39.741323 15664 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 3.
I20250112 20:56:39.741281 15879 consensus_queue.cc:237] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:39.747766 15484 catalog_manager.cc:5526] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 reported cstate change: term changed from 2 to 3, leader changed from 3105649bf63c47b6b4c11216164d816f (127.15.23.130) to f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131). New cstate: current_term: 3 leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: HEALTHY } } }
I20250112 20:56:40.145859 15664 raft_consensus.cc:1270] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:40.147064 15879 consensus_queue.cc:1035] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:40.157603 15589 raft_consensus.cc:1270] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:40.158972 15879 consensus_queue.cc:1035] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:40.257234 15589 raft_consensus.cc:1270] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:40.258248 15879 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:40.268679 15664 raft_consensus.cc:1270] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:40.270151 15879 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250112 20:56:40.337344 15685 debug-util.cc:398] Leaking SignalData structure 0x7b0800133060 after lost signal to thread 15457
W20250112 20:56:40.337988 15685 debug-util.cc:398] Leaking SignalData structure 0x7b0800133740 after lost signal to thread 15520
W20250112 20:56:40.339391 15685 debug-util.cc:398] Leaking SignalData structure 0x7b08001ef420 after lost signal to thread 15762
I20250112 20:56:40.580519 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:40.585371 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:40.585767 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:41.586463 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:41.590885 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:41.591243 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:42.592015 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:42.596832 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:42.597180 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:43.598053 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:43.602689 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:43.603049 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:44.603701 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:44.608224 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:44.608553 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:45.609282 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:45.614132 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:45.614482 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:46.615257 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:46.619829 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:46.620290 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:47.640167 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:47.644693 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:47.645057 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20250112 20:56:48.265326 15610 debug-util.cc:398] Leaking SignalData structure 0x7b08000ec060 after lost signal to thread 15457
W20250112 20:56:48.266039 15610 debug-util.cc:398] Leaking SignalData structure 0x7b08000ec740 after lost signal to thread 15520
W20250112 20:56:48.266724 15610 debug-util.cc:398] Leaking SignalData structure 0x7b08001df480 after lost signal to thread 15613
W20250112 20:56:48.267406 15610 debug-util.cc:398] Leaking SignalData structure 0x7b08000ac3c0 after lost signal to thread 15688
W20250112 20:56:48.268013 15610 debug-util.cc:398] Leaking SignalData structure 0x7b08001e5120 after lost signal to thread 15762
I20250112 20:56:48.645750 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:48.650353 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:48.650708 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:49.678689 15454 auto_leader_rebalancer-test.cc:288] The leader distribution is
I20250112 20:56:49.682945 15589 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
mode: GRACEFUL
new_leader_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:45924
I20250112 20:56:49.683665 15589 raft_consensus.cc:604] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Received request to transfer leadership to TS 3105649bf63c47b6b4c11216164d816f
I20250112 20:56:49.688026 15589 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
mode: GRACEFUL
new_leader_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:45924
I20250112 20:56:49.688447 15589 raft_consensus.cc:604] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Received request to transfer leadership to TS 3105649bf63c47b6b4c11216164d816f
I20250112 20:56:49.692281 15589 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "78a37d408cd1479e93d53989431583f3"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
mode: GRACEFUL
new_leader_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:45924
I20250112 20:56:49.692709 15589 raft_consensus.cc:604] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 LEADER]: Received request to transfer leadership to TS 3105649bf63c47b6b4c11216164d816f
I20250112 20:56:49.696834 15737 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141"
dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
mode: GRACEFUL
new_leader_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:33424
I20250112 20:56:49.697256 15737 raft_consensus.cc:604] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 LEADER]: Received request to transfer leadership to TS 3105649bf63c47b6b4c11216164d816f
I20250112 20:56:49.701251 15737 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731"
dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
mode: GRACEFUL
new_leader_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:33424
I20250112 20:56:49.701650 15737 raft_consensus.cc:604] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 LEADER]: Received request to transfer leadership to TS 3105649bf63c47b6b4c11216164d816f
I20250112 20:56:49.724390 15904 raft_consensus.cc:988] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1: : Instructing follower 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:49.725109 15913 raft_consensus.cc:1076] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Signalling peer 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:49.726888 15664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:44670
I20250112 20:56:49.727329 15664 raft_consensus.cc:491] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:49.727658 15664 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:49.823137 15664 raft_consensus.cc:513] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:49.825830 15664 leader_election.cc:290] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:49.826647 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:49.827579 15589 raft_consensus.cc:3049] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:56:49.827965 15589 raft_consensus.cc:738] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:49.828816 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:49.829321 15737 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:49.828821 15589 consensus_queue.cc:260] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:49.830361 15589 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:49.916643 15904 raft_consensus.cc:988] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1: : Instructing follower 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:49.917117 15913 raft_consensus.cc:1076] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 LEADER]: Signalling peer 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:49.918689 15664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "78a37d408cd1479e93d53989431583f3"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:44670
I20250112 20:56:49.919224 15664 raft_consensus.cc:491] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:49.919584 15664 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:49.929836 15904 raft_consensus.cc:988] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1: : Instructing follower 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:49.930423 15904 raft_consensus.cc:1076] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Signalling peer 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:49.930732 15737 raft_consensus.cc:2463] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:49.931995 15630 leader_election.cc:304] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:49.932251 15663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:44670
I20250112 20:56:49.932816 15663 raft_consensus.cc:491] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:49.933120 15912 raft_consensus.cc:2798] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:56:49.933334 15663 raft_consensus.cc:3054] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:49.935307 15664 raft_consensus.cc:513] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:49.938585 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:49.939160 15736 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:49.941136 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:49.941882 15588 raft_consensus.cc:3049] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:56:49.942253 15588 raft_consensus.cc:738] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:49.943110 15588 consensus_queue.cc:260] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:49.943713 15912 raft_consensus.cc:695] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:49.944322 15588 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:49.930428 15589 raft_consensus.cc:2463] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:49.948630 15664 leader_election.cc:290] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 3 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:49.944710 15912 consensus_queue.cc:237] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:49.962476 15484 catalog_manager.cc:5526] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 1 to 2, leader changed from 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129) to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 2 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } }
I20250112 20:56:50.027520 15920 raft_consensus.cc:988] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5: : Instructing follower 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:50.028064 15920 raft_consensus.cc:1076] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 LEADER]: Signalling peer 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:50.028059 15588 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 3.
I20250112 20:56:50.027431 15663 raft_consensus.cc:513] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.030202 15627 leader_election.cc:304] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
I20250112 20:56:50.030572 15663 leader_election.cc:290] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:50.031365 15912 raft_consensus.cc:2798] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:56:50.031860 15912 raft_consensus.cc:695] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 3 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:50.027398 15736 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 3.
I20250112 20:56:50.032795 15912 consensus_queue.cc:237] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.033205 15588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:50.033731 15588 raft_consensus.cc:3049] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:56:50.034041 15588 raft_consensus.cc:738] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:50.034734 15588 consensus_queue.cc:260] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.035812 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46abc6e120e3421e80afc6dd0d5d84c9" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:50.036072 15588 raft_consensus.cc:3054] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:50.036386 15736 raft_consensus.cc:3054] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:56:50.037530 15663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:44676
I20250112 20:56:50.038237 15663 raft_consensus.cc:491] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:50.038693 15663 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:56:50.053457 15484 catalog_manager.cc:5526] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 2 to 3, leader changed from 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129) to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 3 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:50.070166 15886 raft_consensus.cc:988] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5: : Instructing follower 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:50.070581 15920 raft_consensus.cc:1076] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 LEADER]: Signalling peer 3105649bf63c47b6b4c11216164d816f to start an election
I20250112 20:56:50.071915 15664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
from {username='slave'} at 127.0.0.1:44676
I20250112 20:56:50.072638 15664 raft_consensus.cc:491] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:50.073027 15664 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:50.133172 15663 raft_consensus.cc:513] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.134956 15663 leader_election.cc:290] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 4 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:50.133147 15736 raft_consensus.cc:2463] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:50.137859 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:50.138463 15589 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:56:50.139488 15630 leader_election.cc:304] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:50.139940 15588 raft_consensus.cc:2463] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 2.
I20250112 20:56:50.139726 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:50.140826 15736 raft_consensus.cc:3049] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 LEADER]: Stepping down as leader of term 3
I20250112 20:56:50.141222 15736 raft_consensus.cc:738] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:50.141546 15912 raft_consensus.cc:2798] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:56:50.142066 15912 raft_consensus.cc:695] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:50.142232 15736 consensus_queue.cc:260] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.143136 15912 consensus_queue.cc:237] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.143651 15736 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:56:50.152882 15484 catalog_manager.cc:5526] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 1 to 2, leader changed from 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129) to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 2 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:50.235935 15589 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 4.
I20250112 20:56:50.236243 15736 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 4.
I20250112 20:56:50.235981 15664 raft_consensus.cc:513] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.237589 15630 leader_election.cc:304] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:50.238479 15664 leader_election.cc:290] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 3 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:50.238749 15912 raft_consensus.cc:2798] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 4 FOLLOWER]: Leader election won for term 4
I20250112 20:56:50.239205 15912 raft_consensus.cc:695] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 4 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:50.241114 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:50.241423 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08aeabd9c134e689630f7828f0c4141" candidate_uuid: "3105649bf63c47b6b4c11216164d816f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:50.241710 15736 raft_consensus.cc:3049] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:56:50.242033 15589 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:50.240077 15912 consensus_queue.cc:237] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.242027 15736 raft_consensus.cc:738] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:50.243726 15736 consensus_queue.cc:260] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.244900 15736 raft_consensus.cc:3054] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:50.253249 15484 catalog_manager.cc:5526] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 3 to 4, leader changed from f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131) to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 4 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:50.332937 15589 raft_consensus.cc:2463] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 3.
I20250112 20:56:50.332937 15736 raft_consensus.cc:2463] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3105649bf63c47b6b4c11216164d816f in term 3.
I20250112 20:56:50.334767 15630 leader_election.cc:304] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:50.336079 15912 raft_consensus.cc:2798] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:56:50.336215 15589 raft_consensus.cc:1270] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:50.336830 15912 raft_consensus.cc:695] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 3 LEADER]: Becoming Leader. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:50.337852 15914 consensus_queue.cc:1035] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:56:50.338246 15912 consensus_queue.cc:237] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:50.355228 15484 catalog_manager.cc:5526] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f reported cstate change: term changed from 2 to 3, leader changed from f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131) to 3105649bf63c47b6b4c11216164d816f (127.15.23.130). New cstate: current_term: 3 leader_uuid: "3105649bf63c47b6b4c11216164d816f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:50.375361 15736 raft_consensus.cc:1270] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:50.376698 15912 consensus_queue.cc:1035] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:50.634886 15736 raft_consensus.cc:1270] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:50.635927 15912 consensus_queue.cc:1035] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:50.645079 15589 raft_consensus.cc:1270] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:50.646472 15912 consensus_queue.cc:1035] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:56:50.695183 15589 raft_consensus.cc:1270] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:50.696255 15912 consensus_queue.cc:1035] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:50.704835 15736 raft_consensus.cc:1270] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:56:50.706001 15914 consensus_queue.cc:1035] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:56:50.724814 15736 raft_consensus.cc:1270] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:50.725773 15914 consensus_queue.cc:1035] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:50.733776 15589 raft_consensus.cc:1270] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:50.735095 15926 consensus_queue.cc:1035] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:50.783327 15589 raft_consensus.cc:1270] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 4 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:56:50.784314 15926 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:56:50.793864 15736 raft_consensus.cc:1270] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 4 FOLLOWER]: Refusing update from remote peer 3105649bf63c47b6b4c11216164d816f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:56:50.795046 15926 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:56:52.734709 15454 auto_leader_rebalancer-test.cc:303] The leader distribution is
I20250112 20:56:52.735296 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:52.741356 15664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
mode: GRACEFUL
new_leader_uuid: "546138398ea14db0bcfe85636a6eebe1"
from {username='slave'} at 127.0.0.1:44694
I20250112 20:56:52.741791 15664 raft_consensus.cc:604] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Received request to transfer leadership to TS 546138398ea14db0bcfe85636a6eebe1
I20250112 20:56:52.743734 15664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
mode: GRACEFUL
new_leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:44694
I20250112 20:56:52.744122 15664 raft_consensus.cc:604] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 4 LEADER]: Received request to transfer leadership to TS f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:52.745877 15664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
mode: GRACEFUL
new_leader_uuid: "546138398ea14db0bcfe85636a6eebe1"
from {username='slave'} at 127.0.0.1:44694
I20250112 20:56:52.746248 15664 raft_consensus.cc:604] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Received request to transfer leadership to TS 546138398ea14db0bcfe85636a6eebe1
I20250112 20:56:52.747907 15664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "78a37d408cd1479e93d53989431583f3"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
mode: GRACEFUL
new_leader_uuid: "546138398ea14db0bcfe85636a6eebe1"
from {username='slave'} at 127.0.0.1:44694
I20250112 20:56:52.748279 15664 raft_consensus.cc:604] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 3 LEADER]: Received request to transfer leadership to TS 546138398ea14db0bcfe85636a6eebe1
I20250112 20:56:52.749910 15664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60"
dest_uuid: "3105649bf63c47b6b4c11216164d816f"
mode: GRACEFUL
new_leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:44694
I20250112 20:56:52.750277 15664 raft_consensus.cc:604] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Received request to transfer leadership to TS f0e345cb8d6f447bb560e3e0b204c2c5
I20250112 20:56:52.751109 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20250112 20:56:52.751487 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:52.850458 15940 raft_consensus.cc:988] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f: : Instructing follower f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:52.851416 15926 raft_consensus.cc:1076] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Signalling peer f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:52.852891 15737 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60"
dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:33414
I20250112 20:56:52.853461 15737 raft_consensus.cc:491] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:52.853778 15737 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:52.862634 15737 raft_consensus.cc:513] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:52.865751 15737 leader_election.cc:290] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 3 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:52.867058 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:52.866923 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d2890cbd734733a7cb83d5e91d0d60" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:52.867652 15664 raft_consensus.cc:3049] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:56:52.867827 15589 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:52.867972 15664 raft_consensus.cc:738] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:52.868713 15664 consensus_queue.cc:260] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:52.869796 15664 raft_consensus.cc:3054] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:52.944002 15940 raft_consensus.cc:988] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f: : Instructing follower 546138398ea14db0bcfe85636a6eebe1 to start an election
I20250112 20:56:52.944482 15926 raft_consensus.cc:1076] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Signalling peer 546138398ea14db0bcfe85636a6eebe1 to start an election
I20250112 20:56:52.945797 15588 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
from {username='slave'} at 127.0.0.1:45906
I20250112 20:56:52.946189 15588 raft_consensus.cc:491] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:52.946462 15588 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:52.950237 15940 raft_consensus.cc:988] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f: : Instructing follower 546138398ea14db0bcfe85636a6eebe1 to start an election
I20250112 20:56:52.950647 15926 raft_consensus.cc:1076] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Signalling peer 546138398ea14db0bcfe85636a6eebe1 to start an election
I20250112 20:56:52.951869 15587 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
from {username='slave'} at 127.0.0.1:45906
I20250112 20:56:52.952255 15587 raft_consensus.cc:491] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:52.952498 15587 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:52.967697 15664 raft_consensus.cc:2463] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 3.
I20250112 20:56:52.969405 15701 leader_election.cc:304] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:52.970685 15920 raft_consensus.cc:2798] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:56:52.971041 15589 raft_consensus.cc:2463] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 3.
I20250112 20:56:52.973407 15920 raft_consensus.cc:695] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 LEADER]: Becoming Leader. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:52.974442 15920 consensus_queue.cc:237] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:52.981163 15486 catalog_manager.cc:5526] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 reported cstate change: term changed from 2 to 3, leader changed from 3105649bf63c47b6b4c11216164d816f (127.15.23.130) to f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131). New cstate: current_term: 3 leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: HEALTHY } } }
I20250112 20:56:53.062935 15588 raft_consensus.cc:513] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:53.062916 15587 raft_consensus.cc:513] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.065218 15588 leader_election.cc:290] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 3 election: Requested vote from peers f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:53.065642 15587 leader_election.cc:290] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 3 election: Requested vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:53.066251 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:53.066990 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:53.067677 15736 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:53.066838 15737 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:53.067787 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5984e78e6a014ec6951f87f52ca0c5e4" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:53.068389 15664 raft_consensus.cc:3049] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:56:53.068341 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d8cbcab5504f22a772e3f3f509ef7b" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:53.068779 15664 raft_consensus.cc:738] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:53.068959 15663 raft_consensus.cc:3049] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:56:53.069207 15663 raft_consensus.cc:738] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:53.069437 15664 consensus_queue.cc:260] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:53.069800 15663 consensus_queue.cc:260] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.070573 15664 raft_consensus.cc:3054] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:53.070909 15663 raft_consensus.cc:3054] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:56:53.124234 15940 raft_consensus.cc:988] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: : Instructing follower f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:53.124713 15926 raft_consensus.cc:1076] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 4 LEADER]: Signalling peer f0e345cb8d6f447bb560e3e0b204c2c5 to start an election
I20250112 20:56:53.125968 15738 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731"
dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
from {username='slave'} at 127.0.0.1:33414
I20250112 20:56:53.126422 15738 raft_consensus.cc:491] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:53.126679 15738 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 4 FOLLOWER]: Advancing to term 5
I20250112 20:56:53.167052 15737 raft_consensus.cc:2463] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 3.
I20250112 20:56:53.167279 15736 raft_consensus.cc:2463] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 3.
I20250112 20:56:53.167052 15664 raft_consensus.cc:2463] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 3.
I20250112 20:56:53.168401 15663 raft_consensus.cc:2463] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 3.
I20250112 20:56:53.168678 15555 leader_election.cc:304] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:53.170222 15552 leader_election.cc:304] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, 546138398ea14db0bcfe85636a6eebe1; no voters:
I20250112 20:56:53.171631 15948 raft_consensus.cc:2798] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:56:53.173611 15949 raft_consensus.cc:2798] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:56:53.175209 15949 raft_consensus.cc:695] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 3 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:53.176218 15949 consensus_queue.cc:237] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.176952 15926 raft_consensus.cc:988] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f: : Instructing follower 546138398ea14db0bcfe85636a6eebe1 to start an election
I20250112 20:56:53.176880 15948 raft_consensus.cc:695] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 3 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:53.177560 15940 raft_consensus.cc:1076] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 3 LEADER]: Signalling peer 546138398ea14db0bcfe85636a6eebe1 to start an election
I20250112 20:56:53.177982 15948 consensus_queue.cc:237] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } }
I20250112 20:56:53.178972 15587 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "78a37d408cd1479e93d53989431583f3"
dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
from {username='slave'} at 127.0.0.1:45906
I20250112 20:56:53.179427 15587 raft_consensus.cc:491] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:56:53.179733 15587 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:56:53.186810 15486 catalog_manager.cc:5526] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 2 to 3, leader changed from 3105649bf63c47b6b4c11216164d816f (127.15.23.130) to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 3 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:53.196215 15486 catalog_manager.cc:5526] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 2 to 3, leader changed from 3105649bf63c47b6b4c11216164d816f (127.15.23.130) to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 3 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:53.269852 15587 raft_consensus.cc:513] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.269850 15738 raft_consensus.cc:513] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.271973 15738 leader_election.cc:290] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 5 election: Requested vote from peers 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501), 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339)
I20250112 20:56:53.272018 15587 leader_election.cc:290] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 4 election: Requested vote from peers 3105649bf63c47b6b4c11216164d816f (127.15.23.130:42339), f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131:38129)
I20250112 20:56:53.273711 15738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5"
I20250112 20:56:53.273938 15663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a37d408cd1479e93d53989431583f3" candidate_uuid: "546138398ea14db0bcfe85636a6eebe1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:53.274065 15664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3105649bf63c47b6b4c11216164d816f"
I20250112 20:56:53.274359 15738 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:56:53.274458 15663 raft_consensus.cc:3049] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 3 LEADER]: Stepping down as leader of term 3
I20250112 20:56:53.274580 15664 raft_consensus.cc:3049] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 4 LEADER]: Stepping down as leader of term 4
I20250112 20:56:53.274789 15663 raft_consensus.cc:738] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:53.274964 15664 raft_consensus.cc:738] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3105649bf63c47b6b4c11216164d816f, State: Running, Role: LEADER
I20250112 20:56:53.275046 15587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "355ac9e86ce94946bb77e763ce9c0731" candidate_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "546138398ea14db0bcfe85636a6eebe1"
I20250112 20:56:53.275668 15587 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 4 FOLLOWER]: Advancing to term 5
I20250112 20:56:53.275579 15663 consensus_queue.cc:260] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.275774 15664 consensus_queue.cc:260] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.276866 15663 raft_consensus.cc:3054] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:56:53.277062 15664 raft_consensus.cc:3054] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 4 FOLLOWER]: Advancing to term 5
I20250112 20:56:53.372944 15738 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 4.
I20250112 20:56:53.373184 15587 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 5.
I20250112 20:56:53.372944 15663 raft_consensus.cc:2463] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 546138398ea14db0bcfe85636a6eebe1 in term 4.
I20250112 20:56:53.374236 15555 leader_election.cc:304] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 546138398ea14db0bcfe85636a6eebe1, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:53.374586 15664 raft_consensus.cc:2463] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e345cb8d6f447bb560e3e0b204c2c5 in term 5.
I20250112 20:56:53.375030 15949 raft_consensus.cc:2798] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 4 FOLLOWER]: Leader election won for term 4
I20250112 20:56:53.375510 15949 raft_consensus.cc:695] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 4 LEADER]: Becoming Leader. State: Replica: 546138398ea14db0bcfe85636a6eebe1, State: Running, Role: LEADER
I20250112 20:56:53.375795 15701 leader_election.cc:304] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3105649bf63c47b6b4c11216164d816f, f0e345cb8d6f447bb560e3e0b204c2c5; no voters:
I20250112 20:56:53.376281 15949 consensus_queue.cc:237] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.377036 15920 raft_consensus.cc:2798] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 5 FOLLOWER]: Leader election won for term 5
I20250112 20:56:53.377449 15920 raft_consensus.cc:695] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 5 LEADER]: Becoming Leader. State: Replica: f0e345cb8d6f447bb560e3e0b204c2c5, State: Running, Role: LEADER
I20250112 20:56:53.378265 15920 consensus_queue.cc:237] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } }
I20250112 20:56:53.390650 15486 catalog_manager.cc:5526] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 reported cstate change: term changed from 3 to 4, leader changed from 3105649bf63c47b6b4c11216164d816f (127.15.23.130) to 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129). New cstate: current_term: 4 leader_uuid: "546138398ea14db0bcfe85636a6eebe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: UNKNOWN } } }
I20250112 20:56:53.392884 15484 catalog_manager.cc:5526] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 reported cstate change: term changed from 4 to 5, leader changed from 3105649bf63c47b6b4c11216164d816f (127.15.23.130) to f0e345cb8d6f447bb560e3e0b204c2c5 (127.15.23.131). New cstate: current_term: 5 leader_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 } health_report { overall_health: HEALTHY } } }
I20250112 20:56:53.397689 15588 raft_consensus.cc:1270] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:53.399052 15920 consensus_queue.cc:1035] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:53.413120 15664 raft_consensus.cc:1270] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:53.414714 15942 consensus_queue.cc:1035] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:56:53.651099 15738 raft_consensus.cc:1270] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Refusing update from remote peer 546138398ea14db0bcfe85636a6eebe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:53.652163 15949 consensus_queue.cc:1035] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:53.660048 15664 raft_consensus.cc:1270] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Refusing update from remote peer 546138398ea14db0bcfe85636a6eebe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:53.661788 15949 consensus_queue.cc:1035] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:53.735710 15664 raft_consensus.cc:1270] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Refusing update from remote peer 546138398ea14db0bcfe85636a6eebe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:53.736716 15949 consensus_queue.cc:1035] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:53.744905 15738 raft_consensus.cc:1270] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Refusing update from remote peer 546138398ea14db0bcfe85636a6eebe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:56:53.746217 15948 consensus_queue.cc:1035] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:56:53.752449 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:53.758396 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:53.758757 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:53.825297 15738 raft_consensus.cc:1270] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 4 FOLLOWER]: Refusing update from remote peer 546138398ea14db0bcfe85636a6eebe1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:56:53.826274 15948 consensus_queue.cc:1035] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e345cb8d6f447bb560e3e0b204c2c5" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:56:53.833479 15664 raft_consensus.cc:1270] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 4 FOLLOWER]: Refusing update from remote peer 546138398ea14db0bcfe85636a6eebe1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:56:53.835405 15966 consensus_queue.cc:1035] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:56:53.887990 15588 raft_consensus.cc:1270] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 5 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20250112 20:56:53.888931 15920 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "546138398ea14db0bcfe85636a6eebe1" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:56:53.896886 15664 raft_consensus.cc:1270] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 5 FOLLOWER]: Refusing update from remote peer f0e345cb8d6f447bb560e3e0b204c2c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20250112 20:56:53.898857 15942 consensus_queue.cc:1035] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3105649bf63c47b6b4c11216164d816f" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:56:54.759507 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:54.763906 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:54.764276 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:55.765003 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:55.770258 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:55.770576 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:56.771384 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:56.775813 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:56.776232 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:57.776923 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:57.781481 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:57.781802 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:58.782459 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:58.787597 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:58.787931 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:56:59.788702 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:56:59.793110 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:56:59.793438 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:00.794212 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:00.798777 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:00.799103 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:01.799825 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:01.805397 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:01.805812 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:02.828972 15454 auto_leader_rebalancer-test.cc:314] The leader distribution is
I20250112 20:57:02.829514 15454 tablet_server.cc:178] TabletServer@127.15.23.129:0 shutting down...
W20250112 20:57:02.841356 15701 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250112 20:57:02.863471 15701 consensus_peers.cc:487] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 -> Peer 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501): Couldn't send request to peer 546138398ea14db0bcfe85636a6eebe1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:57:02.876823 15701 consensus_peers.cc:487] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 -> Peer 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501): Couldn't send request to peer 546138398ea14db0bcfe85636a6eebe1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:02.892074 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:57:02.893258 15454 tablet_replica.cc:331] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
I20250112 20:57:02.895107 15454 raft_consensus.cc:2238] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:02.896409 15454 raft_consensus.cc:2267] T 355ac9e86ce94946bb77e763ce9c0731 P 546138398ea14db0bcfe85636a6eebe1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:02.899919 15454 tablet_replica.cc:331] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
I20250112 20:57:02.900771 15454 raft_consensus.cc:2238] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 4 LEADER]: Raft consensus shutting down.
I20250112 20:57:02.901995 15454 raft_consensus.cc:2267] T 78a37d408cd1479e93d53989431583f3 P 546138398ea14db0bcfe85636a6eebe1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:02.903770 15454 tablet_replica.cc:331] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
W20250112 20:57:02.904088 15627 consensus_peers.cc:487] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f -> Peer 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501): Couldn't send request to peer 546138398ea14db0bcfe85636a6eebe1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:02.904332 15454 raft_consensus.cc:2238] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:02.904846 15454 raft_consensus.cc:2267] T 46abc6e120e3421e80afc6dd0d5d84c9 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:02.907079 15454 tablet_replica.cc:331] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
I20250112 20:57:02.907517 15454 raft_consensus.cc:2238] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:02.907932 15454 raft_consensus.cc:2267] T 0f7944602db44c8794b4bebaaf1f453a P 546138398ea14db0bcfe85636a6eebe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:02.909652 15454 tablet_replica.cc:331] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
I20250112 20:57:02.910187 15454 raft_consensus.cc:2238] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:02.911091 15454 raft_consensus.cc:2267] T 5984e78e6a014ec6951f87f52ca0c5e4 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:02.912576 15454 tablet_replica.cc:331] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
I20250112 20:57:02.913045 15454 raft_consensus.cc:2238] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:02.913465 15454 raft_consensus.cc:2267] T b38b65b2ea0944e2a1c04e31efb36270 P 546138398ea14db0bcfe85636a6eebe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:02.915095 15454 tablet_replica.cc:331] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
I20250112 20:57:02.915526 15454 raft_consensus.cc:2238] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:02.916518 15454 raft_consensus.cc:2267] T 13d8cbcab5504f22a772e3f3f509ef7b P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:02.918052 15454 tablet_replica.cc:331] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
I20250112 20:57:02.918473 15454 raft_consensus.cc:2238] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:02.918926 15454 raft_consensus.cc:2267] T a08aeabd9c134e689630f7828f0c4141 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:02.920778 15454 tablet_replica.cc:331] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1: stopping tablet replica
I20250112 20:57:02.921552 15454 raft_consensus.cc:2238] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:02.922339 15454 raft_consensus.cc:2267] T e0d2890cbd734733a7cb83d5e91d0d60 P 546138398ea14db0bcfe85636a6eebe1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20250112 20:57:02.992730 15627 consensus_peers.cc:487] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f -> Peer 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501): Couldn't send request to peer 546138398ea14db0bcfe85636a6eebe1. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:03.018188 15454 tablet_server.cc:195] TabletServer@127.15.23.129:0 shutdown complete.
I20250112 20:57:03.038173 15454 tablet_server.cc:178] TabletServer@127.15.23.130:0 shutting down...
I20250112 20:57:03.063616 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:57:03.064265 15454 tablet_replica.cc:331] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.064906 15454 raft_consensus.cc:2238] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:57:03.065595 15454 raft_consensus.cc:2267] T 0f7944602db44c8794b4bebaaf1f453a P 3105649bf63c47b6b4c11216164d816f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.067094 15454 tablet_replica.cc:331] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.067560 15454 raft_consensus.cc:2238] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.067922 15454 raft_consensus.cc:2267] T 13d8cbcab5504f22a772e3f3f509ef7b P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.069357 15454 tablet_replica.cc:331] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.069850 15454 raft_consensus.cc:2238] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:03.070505 15454 raft_consensus.cc:2267] T a08aeabd9c134e689630f7828f0c4141 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.071854 15454 tablet_replica.cc:331] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.072266 15454 raft_consensus.cc:2238] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.072640 15454 raft_consensus.cc:2267] T e0d2890cbd734733a7cb83d5e91d0d60 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.074007 15454 tablet_replica.cc:331] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.074440 15454 raft_consensus.cc:2238] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.074782 15454 raft_consensus.cc:2267] T b38b65b2ea0944e2a1c04e31efb36270 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.076179 15454 tablet_replica.cc:331] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.076655 15454 raft_consensus.cc:2238] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.077064 15454 raft_consensus.cc:2267] T 78a37d408cd1479e93d53989431583f3 P 3105649bf63c47b6b4c11216164d816f [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.078442 15454 tablet_replica.cc:331] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.078864 15454 raft_consensus.cc:2238] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:57:03.079635 15454 raft_consensus.cc:2267] T 46abc6e120e3421e80afc6dd0d5d84c9 P 3105649bf63c47b6b4c11216164d816f [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.081190 15454 tablet_replica.cc:331] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.081563 15454 raft_consensus.cc:2238] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 5 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.081933 15454 raft_consensus.cc:2267] T 355ac9e86ce94946bb77e763ce9c0731 P 3105649bf63c47b6b4c11216164d816f [term 5 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.083317 15454 tablet_replica.cc:331] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f: stopping tablet replica
I20250112 20:57:03.083827 15454 raft_consensus.cc:2238] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.084430 15454 raft_consensus.cc:2267] T 5984e78e6a014ec6951f87f52ca0c5e4 P 3105649bf63c47b6b4c11216164d816f [term 3 FOLLOWER]: Raft consensus is shut down!
W20250112 20:57:03.113220 15701 consensus_peers.cc:487] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 -> Peer 546138398ea14db0bcfe85636a6eebe1 (127.15.23.129:44501): Couldn't send request to peer 546138398ea14db0bcfe85636a6eebe1. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:03.126806 15454 tablet_server.cc:195] TabletServer@127.15.23.130:0 shutdown complete.
I20250112 20:57:03.143055 15454 tablet_server.cc:178] TabletServer@127.15.23.131:0 shutting down...
I20250112 20:57:03.166374 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:57:03.166932 15454 tablet_replica.cc:331] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.167461 15454 raft_consensus.cc:2238] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:57:03.168210 15454 raft_consensus.cc:2267] T b38b65b2ea0944e2a1c04e31efb36270 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.169917 15454 tablet_replica.cc:331] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.170351 15454 raft_consensus.cc:2238] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.170708 15454 raft_consensus.cc:2267] T a08aeabd9c134e689630f7828f0c4141 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.172130 15454 tablet_replica.cc:331] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.172657 15454 raft_consensus.cc:2238] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.173070 15454 raft_consensus.cc:2267] T 13d8cbcab5504f22a772e3f3f509ef7b P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.174438 15454 tablet_replica.cc:331] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.174849 15454 raft_consensus.cc:2238] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.175158 15454 raft_consensus.cc:2267] T 46abc6e120e3421e80afc6dd0d5d84c9 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.176522 15454 tablet_replica.cc:331] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.176985 15454 raft_consensus.cc:2238] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 5 LEADER]: Raft consensus shutting down.
I20250112 20:57:03.177759 15454 raft_consensus.cc:2267] T 355ac9e86ce94946bb77e763ce9c0731 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.179248 15454 tablet_replica.cc:331] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.179679 15454 raft_consensus.cc:2238] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.180058 15454 raft_consensus.cc:2267] T 78a37d408cd1479e93d53989431583f3 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.181452 15454 tablet_replica.cc:331] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.181833 15454 raft_consensus.cc:2238] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:03.182475 15454 raft_consensus.cc:2267] T e0d2890cbd734733a7cb83d5e91d0d60 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.183740 15454 tablet_replica.cc:331] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.184145 15454 raft_consensus.cc:2238] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.184542 15454 raft_consensus.cc:2267] T 0f7944602db44c8794b4bebaaf1f453a P f0e345cb8d6f447bb560e3e0b204c2c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.185946 15454 tablet_replica.cc:331] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5: stopping tablet replica
I20250112 20:57:03.186298 15454 raft_consensus.cc:2238] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:03.186586 15454 raft_consensus.cc:2267] T 5984e78e6a014ec6951f87f52ca0c5e4 P f0e345cb8d6f447bb560e3e0b204c2c5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.211751 15454 tablet_server.cc:195] TabletServer@127.15.23.131:0 shutdown complete.
I20250112 20:57:03.228550 15454 master.cc:537] Master@127.15.23.190:42983 shutting down...
I20250112 20:57:03.348991 15454 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:57:03.349610 15454 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:03.349881 15454 tablet_replica.cc:331] T 00000000000000000000000000000000 P 0c7eec2e6c784725bdaec7ec25aaa4af: stopping tablet replica
I20250112 20:57:03.369359 15454 master.cc:559] Master@127.15.23.190:42983 shutdown complete.
3105649bf63c47b6b4c11216164d816f 4
546138398ea14db0bcfe85636a6eebe1 4
f0e345cb8d6f447bb560e3e0b204c2c5 1
3105649bf63c47b6b4c11216164d816f 3
546138398ea14db0bcfe85636a6eebe1 3
f0e345cb8d6f447bb560e3e0b204c2c5 3
3105649bf63c47b6b4c11216164d816f 8
546138398ea14db0bcfe85636a6eebe1 0
f0e345cb8d6f447bb560e3e0b204c2c5 1
3105649bf63c47b6b4c11216164d816f 3
546138398ea14db0bcfe85636a6eebe1 3
f0e345cb8d6f447bb560e3e0b204c2c5 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (30657 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20250112 20:57:03.409308 15454 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.23.190:39165
I20250112 20:57:03.410351 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:57:03.415869 15982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:03.416610 15983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:03.417282 15985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:03.418325 15454 server_base.cc:1034] running on GCE node
I20250112 20:57:03.419301 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:03.419487 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:03.419591 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715423419581 us; error 0 us; skew 500 ppm
I20250112 20:57:03.420106 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:03.426239 15454 webserver.cc:458] Webserver started at http://127.15.23.190:39451/ using document root <none> and password file <none>
I20250112 20:57:03.426734 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:03.426903 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:03.427173 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:03.428301 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/master-0-root/instance:
uuid: "f6eff2bdc6564f2898d116fc5d0f4a00"
format_stamp: "Formatted at 2025-01-12 20:57:03 on dist-test-slave-m2nm"
I20250112 20:57:03.432642 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20250112 20:57:03.435801 15990 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.436475 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250112 20:57:03.436760 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/master-0-root
uuid: "f6eff2bdc6564f2898d116fc5d0f4a00"
format_stamp: "Formatted at 2025-01-12 20:57:03 on dist-test-slave-m2nm"
I20250112 20:57:03.437041 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:03.460031 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:03.461230 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:03.496428 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.190:39165
I20250112 20:57:03.496507 16041 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.190:39165 every 8 connection(s)
I20250112 20:57:03.500068 16042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:03.508528 16042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00: Bootstrap starting.
I20250112 20:57:03.512883 16042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:03.516674 16042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00: No bootstrap required, opened a new log
I20250112 20:57:03.518788 16042 raft_consensus.cc:357] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eff2bdc6564f2898d116fc5d0f4a00" member_type: VOTER }
I20250112 20:57:03.519229 16042 raft_consensus.cc:383] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:03.519470 16042 raft_consensus.cc:738] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6eff2bdc6564f2898d116fc5d0f4a00, State: Initialized, Role: FOLLOWER
I20250112 20:57:03.519976 16042 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eff2bdc6564f2898d116fc5d0f4a00" member_type: VOTER }
I20250112 20:57:03.520395 16042 raft_consensus.cc:397] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 20:57:03.520632 16042 raft_consensus.cc:491] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 20:57:03.520891 16042 raft_consensus.cc:3054] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:03.524454 16042 raft_consensus.cc:513] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eff2bdc6564f2898d116fc5d0f4a00" member_type: VOTER }
I20250112 20:57:03.524945 16042 leader_election.cc:304] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6eff2bdc6564f2898d116fc5d0f4a00; no voters:
I20250112 20:57:03.525993 16042 leader_election.cc:290] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 20:57:03.526332 16045 raft_consensus.cc:2798] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:03.527477 16045 raft_consensus.cc:695] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 1 LEADER]: Becoming Leader. State: Replica: f6eff2bdc6564f2898d116fc5d0f4a00, State: Running, Role: LEADER
I20250112 20:57:03.528128 16045 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eff2bdc6564f2898d116fc5d0f4a00" member_type: VOTER }
I20250112 20:57:03.528787 16042 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [sys.catalog]: configured and running, proceeding with master startup.
I20250112 20:57:03.534459 16046 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6eff2bdc6564f2898d116fc5d0f4a00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eff2bdc6564f2898d116fc5d0f4a00" member_type: VOTER } }
I20250112 20:57:03.534529 16047 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6eff2bdc6564f2898d116fc5d0f4a00. Latest consensus state: current_term: 1 leader_uuid: "f6eff2bdc6564f2898d116fc5d0f4a00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eff2bdc6564f2898d116fc5d0f4a00" member_type: VOTER } }
I20250112 20:57:03.535311 16046 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [sys.catalog]: This master's current role is: LEADER
I20250112 20:57:03.535372 16047 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [sys.catalog]: This master's current role is: LEADER
I20250112 20:57:03.537252 16055 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 20:57:03.542565 16055 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 20:57:03.543591 15454 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250112 20:57:03.550498 16055 catalog_manager.cc:1348] Generated new cluster ID: 185c98f3b5854740a7b406f483d0406d
I20250112 20:57:03.550730 16055 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 20:57:03.568204 16055 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 20:57:03.569341 16055 catalog_manager.cc:1505] Loading token signing keys...
I20250112 20:57:03.590090 16055 catalog_manager.cc:5899] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00: Generated new TSK 0
I20250112 20:57:03.590600 16055 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 20:57:03.609918 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:57:03.616050 16063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:03.617373 16064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:03.618834 15454 server_base.cc:1034] running on GCE node
W20250112 20:57:03.619468 16066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:03.620293 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:03.620477 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:03.620638 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715423620621 us; error 0 us; skew 500 ppm
I20250112 20:57:03.621101 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:03.623364 15454 webserver.cc:458] Webserver started at http://127.15.23.129:36389/ using document root <none> and password file <none>
I20250112 20:57:03.623803 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:03.623960 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:03.624162 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:03.625190 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-0-root/instance:
uuid: "6e59a26136604032abe0758f79a8562a"
format_stamp: "Formatted at 2025-01-12 20:57:03 on dist-test-slave-m2nm"
I20250112 20:57:03.629448 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20250112 20:57:03.632407 16071 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.633111 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250112 20:57:03.633361 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-0-root
uuid: "6e59a26136604032abe0758f79a8562a"
format_stamp: "Formatted at 2025-01-12 20:57:03 on dist-test-slave-m2nm"
I20250112 20:57:03.633605 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:03.660270 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:03.661434 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:03.662711 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:57:03.664913 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:57:03.665088 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.665324 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:57:03.665464 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.701416 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.129:38359
I20250112 20:57:03.701495 16133 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.129:38359 every 8 connection(s)
I20250112 20:57:03.705422 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:57:03.713094 16138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:03.714241 16139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:03.717808 16134 heartbeater.cc:346] Connected to a master server at 127.15.23.190:39165
W20250112 20:57:03.718106 16141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:03.718130 16134 heartbeater.cc:463] Registering TS with master...
I20250112 20:57:03.719033 16134 heartbeater.cc:510] Master 127.15.23.190:39165 requested a full tablet report, sending...
I20250112 20:57:03.719311 15454 server_base.cc:1034] running on GCE node
I20250112 20:57:03.720288 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:03.720530 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:03.720749 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715423720724 us; error 0 us; skew 500 ppm
I20250112 20:57:03.721159 16007 ts_manager.cc:194] Registered new tserver with Master: 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359)
I20250112 20:57:03.721249 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:03.723016 16007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44922
I20250112 20:57:03.723804 15454 webserver.cc:458] Webserver started at http://127.15.23.130:34763/ using document root <none> and password file <none>
I20250112 20:57:03.724349 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:03.724514 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:03.724835 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:03.725970 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-1-root/instance:
uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
format_stamp: "Formatted at 2025-01-12 20:57:03 on dist-test-slave-m2nm"
I20250112 20:57:03.729956 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250112 20:57:03.732831 16146 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.733461 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250112 20:57:03.733688 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-1-root
uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
format_stamp: "Formatted at 2025-01-12 20:57:03 on dist-test-slave-m2nm"
I20250112 20:57:03.733944 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:03.755844 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:03.756983 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:03.758256 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:57:03.760241 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:57:03.760413 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.760632 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:57:03.760779 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.797756 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.130:36113
I20250112 20:57:03.797842 16208 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.130:36113 every 8 connection(s)
I20250112 20:57:03.803246 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:57:03.812424 16212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:03.815110 16213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:03.816241 16209 heartbeater.cc:346] Connected to a master server at 127.15.23.190:39165
I20250112 20:57:03.816718 16209 heartbeater.cc:463] Registering TS with master...
I20250112 20:57:03.817461 16209 heartbeater.cc:510] Master 127.15.23.190:39165 requested a full tablet report, sending...
I20250112 20:57:03.817777 15454 server_base.cc:1034] running on GCE node
W20250112 20:57:03.819048 16215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:03.819525 16007 ts_manager.cc:194] Registered new tserver with Master: d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:03.819993 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:03.820250 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:03.820452 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715423820435 us; error 0 us; skew 500 ppm
I20250112 20:57:03.821185 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:03.821314 16007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44926
I20250112 20:57:03.823836 15454 webserver.cc:458] Webserver started at http://127.15.23.131:42711/ using document root <none> and password file <none>
I20250112 20:57:03.824270 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:03.824416 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:03.824641 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:03.825681 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-2-root/instance:
uuid: "2f592694e8d94b6b9336474b40fa220c"
format_stamp: "Formatted at 2025-01-12 20:57:03 on dist-test-slave-m2nm"
I20250112 20:57:03.829675 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250112 20:57:03.832477 16220 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.833113 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250112 20:57:03.833390 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-2-root
uuid: "2f592694e8d94b6b9336474b40fa220c"
format_stamp: "Formatted at 2025-01-12 20:57:03 on dist-test-slave-m2nm"
I20250112 20:57:03.833642 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1736715392595610-15454-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:03.850109 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:03.851209 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:03.852499 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:57:03.854502 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:57:03.854681 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.854916 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:57:03.855067 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:03.890548 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.131:34743
I20250112 20:57:03.890666 16282 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.131:34743 every 8 connection(s)
I20250112 20:57:03.903389 16283 heartbeater.cc:346] Connected to a master server at 127.15.23.190:39165
I20250112 20:57:03.903729 16283 heartbeater.cc:463] Registering TS with master...
I20250112 20:57:03.904392 16283 heartbeater.cc:510] Master 127.15.23.190:39165 requested a full tablet report, sending...
I20250112 20:57:03.906109 16007 ts_manager.cc:194] Registered new tserver with Master: 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:03.906759 15454 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013113625s
I20250112 20:57:03.907557 16007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44932
I20250112 20:57:03.909770 15454 test_util.cc:274] Using random seed: -1846868117
I20250112 20:57:03.929908 16007 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44946:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250112 20:57:03.932653 16007 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250112 20:57:03.997591 16094 tablet_service.cc:1467] Processing CreateTablet for tablet 429a8b20b2184c82a2222da7025ba714 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250112 20:57:03.998983 16094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429a8b20b2184c82a2222da7025ba714. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.002065 16098 tablet_service.cc:1467] Processing CreateTablet for tablet 85ad275e24e24a3eafa4192cc2e00929 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250112 20:57:04.003417 16098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85ad275e24e24a3eafa4192cc2e00929. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.003427 16097 tablet_service.cc:1467] Processing CreateTablet for tablet 986bdc5230e9412092af88b2f5d1fb5b (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250112 20:57:04.004818 16097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986bdc5230e9412092af88b2f5d1fb5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.009820 16099 tablet_service.cc:1467] Processing CreateTablet for tablet d969c6c299574510b416acf29125bbfc (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250112 20:57:04.011122 16099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d969c6c299574510b416acf29125bbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.012467 16096 tablet_service.cc:1467] Processing CreateTablet for tablet 51f70e21e6804833a0e56fd3436032cc (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250112 20:57:04.014081 16096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51f70e21e6804833a0e56fd3436032cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.017813 16095 tablet_service.cc:1467] Processing CreateTablet for tablet daed7450d0c64fc784f5b2c8cd139203 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250112 20:57:04.019013 16095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daed7450d0c64fc784f5b2c8cd139203. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.021350 16093 tablet_service.cc:1467] Processing CreateTablet for tablet 45f78ae8287d494781c3d4b9fae23e2f (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250112 20:57:04.021548 16092 tablet_service.cc:1467] Processing CreateTablet for tablet cb464103f740429396ec85635d9853ca (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250112 20:57:04.022588 16093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45f78ae8287d494781c3d4b9fae23e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.023267 16091 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac4a46114fc47ceb7f96990ab3bfa33 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250112 20:57:04.025203 16090 tablet_service.cc:1467] Processing CreateTablet for tablet 1e32cc98919f4a5fad3816a08f4a4b18 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250112 20:57:04.026500 16091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ac4a46114fc47ceb7f96990ab3bfa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.030215 16090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e32cc98919f4a5fad3816a08f4a4b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.033239 16092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb464103f740429396ec85635d9853ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.045876 16172 tablet_service.cc:1467] Processing CreateTablet for tablet 986bdc5230e9412092af88b2f5d1fb5b (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250112 20:57:04.046131 16173 tablet_service.cc:1467] Processing CreateTablet for tablet 85ad275e24e24a3eafa4192cc2e00929 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250112 20:57:04.047176 16172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986bdc5230e9412092af88b2f5d1fb5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.048230 16174 tablet_service.cc:1467] Processing CreateTablet for tablet d969c6c299574510b416acf29125bbfc (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250112 20:57:04.049487 16174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d969c6c299574510b416acf29125bbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.049576 16171 tablet_service.cc:1467] Processing CreateTablet for tablet 51f70e21e6804833a0e56fd3436032cc (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250112 20:57:04.050828 16171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51f70e21e6804833a0e56fd3436032cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.048226 16170 tablet_service.cc:1467] Processing CreateTablet for tablet daed7450d0c64fc784f5b2c8cd139203 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250112 20:57:04.052451 16170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daed7450d0c64fc784f5b2c8cd139203. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.055285 16168 tablet_service.cc:1467] Processing CreateTablet for tablet 45f78ae8287d494781c3d4b9fae23e2f (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250112 20:57:04.057786 16167 tablet_service.cc:1467] Processing CreateTablet for tablet cb464103f740429396ec85635d9853ca (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250112 20:57:04.061708 16165 tablet_service.cc:1467] Processing CreateTablet for tablet 1e32cc98919f4a5fad3816a08f4a4b18 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250112 20:57:04.060086 16166 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac4a46114fc47ceb7f96990ab3bfa33 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250112 20:57:04.056478 16168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45f78ae8287d494781c3d4b9fae23e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.064384 16165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e32cc98919f4a5fad3816a08f4a4b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.064335 16169 tablet_service.cc:1467] Processing CreateTablet for tablet 429a8b20b2184c82a2222da7025ba714 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250112 20:57:04.065626 16169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429a8b20b2184c82a2222da7025ba714. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.068877 16166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ac4a46114fc47ceb7f96990ab3bfa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.072451 16173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85ad275e24e24a3eafa4192cc2e00929. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.076958 16167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb464103f740429396ec85635d9853ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.111694 16297 tablet_bootstrap.cc:492] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.113904 16298 tablet_bootstrap.cc:492] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.118196 16297 tablet_bootstrap.cc:654] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.121132 16247 tablet_service.cc:1467] Processing CreateTablet for tablet 85ad275e24e24a3eafa4192cc2e00929 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250112 20:57:04.121487 16298 tablet_bootstrap.cc:654] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.121672 16246 tablet_service.cc:1467] Processing CreateTablet for tablet 986bdc5230e9412092af88b2f5d1fb5b (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250112 20:57:04.122473 16247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85ad275e24e24a3eafa4192cc2e00929. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.123888 16245 tablet_service.cc:1467] Processing CreateTablet for tablet 51f70e21e6804833a0e56fd3436032cc (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250112 20:57:04.125123 16245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51f70e21e6804833a0e56fd3436032cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.125319 16248 tablet_service.cc:1467] Processing CreateTablet for tablet d969c6c299574510b416acf29125bbfc (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250112 20:57:04.126474 16248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d969c6c299574510b416acf29125bbfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.132087 16244 tablet_service.cc:1467] Processing CreateTablet for tablet daed7450d0c64fc784f5b2c8cd139203 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250112 20:57:04.132879 16242 tablet_service.cc:1467] Processing CreateTablet for tablet 45f78ae8287d494781c3d4b9fae23e2f (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250112 20:57:04.133683 16244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daed7450d0c64fc784f5b2c8cd139203. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.135236 16241 tablet_service.cc:1467] Processing CreateTablet for tablet cb464103f740429396ec85635d9853ca (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250112 20:57:04.136436 16241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb464103f740429396ec85635d9853ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.136958 16243 tablet_service.cc:1467] Processing CreateTablet for tablet 429a8b20b2184c82a2222da7025ba714 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250112 20:57:04.138550 16243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429a8b20b2184c82a2222da7025ba714. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.140918 16246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986bdc5230e9412092af88b2f5d1fb5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.144968 16242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45f78ae8287d494781c3d4b9fae23e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.150362 16240 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac4a46114fc47ceb7f96990ab3bfa33 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250112 20:57:04.151579 16240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ac4a46114fc47ceb7f96990ab3bfa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.159092 16301 tablet_bootstrap.cc:492] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.162042 16239 tablet_service.cc:1467] Processing CreateTablet for tablet 1e32cc98919f4a5fad3816a08f4a4b18 (DEFAULT_TABLE table=test-workload [id=3ca0b1e71dd1452b8fa11d8aceff8a71]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250112 20:57:04.163004 16239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e32cc98919f4a5fad3816a08f4a4b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:04.180444 16298 tablet_bootstrap.cc:492] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.180964 16298 ts_tablet_manager.cc:1397] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.067s user 0.027s sys 0.011s
I20250112 20:57:04.183607 16298 raft_consensus.cc:357] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.184334 16298 raft_consensus.cc:383] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.184685 16298 raft_consensus.cc:738] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.190009 16298 consensus_queue.cc:260] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.196735 16209 heartbeater.cc:502] Master 127.15.23.190:39165 was elected leader, sending a full tablet report...
I20250112 20:57:04.197289 16297 tablet_bootstrap.cc:492] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.197408 16298 ts_tablet_manager.cc:1428] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.016s user 0.008s sys 0.004s
I20250112 20:57:04.197755 16297 ts_tablet_manager.cc:1397] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.086s user 0.016s sys 0.008s
I20250112 20:57:04.198292 16298 tablet_bootstrap.cc:492] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.204553 16298 tablet_bootstrap.cc:654] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.205037 16301 tablet_bootstrap.cc:654] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.200493 16297 raft_consensus.cc:357] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.208966 16297 raft_consensus.cc:383] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.209358 16297 raft_consensus.cc:738] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.210312 16297 consensus_queue.cc:260] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.213526 16134 heartbeater.cc:502] Master 127.15.23.190:39165 was elected leader, sending a full tablet report...
I20250112 20:57:04.215061 16297 ts_tablet_manager.cc:1428] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.017s user 0.006s sys 0.002s
I20250112 20:57:04.215871 16297 tablet_bootstrap.cc:492] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.216830 16301 tablet_bootstrap.cc:492] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.217337 16301 ts_tablet_manager.cc:1397] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.059s user 0.013s sys 0.000s
I20250112 20:57:04.220460 16298 tablet_bootstrap.cc:492] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.220377 16301 raft_consensus.cc:357] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.221041 16298 ts_tablet_manager.cc:1397] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.009s
I20250112 20:57:04.221360 16301 raft_consensus.cc:383] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.221743 16301 raft_consensus.cc:738] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.222083 16297 tablet_bootstrap.cc:654] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.222612 16301 consensus_queue.cc:260] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.228940 16298 raft_consensus.cc:357] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.229921 16298 raft_consensus.cc:383] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.230299 16298 raft_consensus.cc:738] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.231312 16298 consensus_queue.cc:260] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.234681 16298 ts_tablet_manager.cc:1428] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20250112 20:57:04.236514 16298 tablet_bootstrap.cc:492] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.255462 16297 tablet_bootstrap.cc:492] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.303145 16283 heartbeater.cc:502] Master 127.15.23.190:39165 was elected leader, sending a full tablet report...
I20250112 20:57:04.359380 16298 tablet_bootstrap.cc:654] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.365821 16298 tablet_bootstrap.cc:492] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.367885 16298 ts_tablet_manager.cc:1397] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.132s user 0.014s sys 0.001s
I20250112 20:57:04.372428 16298 raft_consensus.cc:357] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.373510 16298 raft_consensus.cc:383] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.373878 16298 raft_consensus.cc:738] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.374541 16298 consensus_queue.cc:260] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.376919 16298 ts_tablet_manager.cc:1428] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20250112 20:57:04.377812 16298 tablet_bootstrap.cc:492] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.383049 16297 ts_tablet_manager.cc:1397] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.167s user 0.006s sys 0.021s
I20250112 20:57:04.386229 16298 tablet_bootstrap.cc:654] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.385919 16297 raft_consensus.cc:357] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.386670 16297 raft_consensus.cc:383] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.386970 16297 raft_consensus.cc:738] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.387530 16297 consensus_queue.cc:260] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.390319 16297 ts_tablet_manager.cc:1428] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20250112 20:57:04.390820 16298 tablet_bootstrap.cc:492] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.391168 16297 tablet_bootstrap.cc:492] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.391247 16298 ts_tablet_manager.cc:1397] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.005s
I20250112 20:57:04.393553 16298 raft_consensus.cc:357] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.394205 16298 raft_consensus.cc:383] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.394470 16298 raft_consensus.cc:738] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.395025 16298 consensus_queue.cc:260] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.396998 16298 ts_tablet_manager.cc:1428] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20250112 20:57:04.398257 16297 tablet_bootstrap.cc:654] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.398751 16298 tablet_bootstrap.cc:492] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.402858 16305 raft_consensus.cc:491] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.403661 16298 tablet_bootstrap.cc:654] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.403409 16305 raft_consensus.cc:513] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.406257 16305 leader_election.cc:290] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.408869 16298 tablet_bootstrap.cc:492] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.408881 16297 tablet_bootstrap.cc:492] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.409415 16298 ts_tablet_manager.cc:1397] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20250112 20:57:04.409479 16297 ts_tablet_manager.cc:1397] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20250112 20:57:04.412195 16298 raft_consensus.cc:357] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.412855 16298 raft_consensus.cc:383] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.413271 16298 raft_consensus.cc:738] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.412820 16297 raft_consensus.cc:357] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.413833 16297 raft_consensus.cc:383] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.413844 16298 consensus_queue.cc:260] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.414366 16297 raft_consensus.cc:738] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.415272 16297 consensus_queue.cc:260] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.416421 16298 ts_tablet_manager.cc:1428] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250112 20:57:04.417387 16298 tablet_bootstrap.cc:492] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.418514 16297 ts_tablet_manager.cc:1428] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20250112 20:57:04.419673 16297 tablet_bootstrap.cc:492] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.423365 16298 tablet_bootstrap.cc:654] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.427017 16297 tablet_bootstrap.cc:654] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.437749 16301 ts_tablet_manager.cc:1428] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.220s user 0.023s sys 0.024s
I20250112 20:57:04.438946 16301 tablet_bootstrap.cc:492] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.439157 16297 tablet_bootstrap.cc:492] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.439738 16297 ts_tablet_manager.cc:1397] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.004s
I20250112 20:57:04.440953 16306 raft_consensus.cc:491] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.441490 16306 raft_consensus.cc:513] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.442986 16297 raft_consensus.cc:357] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.443831 16297 raft_consensus.cc:383] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.444164 16297 raft_consensus.cc:738] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.444986 16297 consensus_queue.cc:260] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.445561 16306 leader_election.cc:290] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:04.448110 16297 ts_tablet_manager.cc:1428] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20250112 20:57:04.449239 16297 tablet_bootstrap.cc:492] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.456192 16302 raft_consensus.cc:491] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.456665 16297 tablet_bootstrap.cc:654] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.456651 16302 raft_consensus.cc:513] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.459514 16302 leader_election.cc:290] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.502312 16305 raft_consensus.cc:491] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.502972 16305 raft_consensus.cc:513] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.505456 16305 leader_election.cc:290] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.532851 16305 raft_consensus.cc:491] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.533368 16305 raft_consensus.cc:513] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.535176 16305 leader_election.cc:290] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
W20250112 20:57:04.547487 16054 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:04.558374 16302 raft_consensus.cc:491] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.558821 16302 raft_consensus.cc:513] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.560108 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:04.560710 16184 raft_consensus.cc:2463] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 0.
I20250112 20:57:04.560810 16302 leader_election.cc:290] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.561717 16224 leader_election.cc:304] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.562428 16306 raft_consensus.cc:2798] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:04.562738 16306 raft_consensus.cc:491] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:04.563024 16306 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.566110 16298 tablet_bootstrap.cc:492] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.566450 16298 ts_tablet_manager.cc:1397] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.149s user 0.033s sys 0.053s
I20250112 20:57:04.568495 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:04.568670 16298 raft_consensus.cc:357] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.569077 16306 raft_consensus.cc:513] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.569291 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:04.569952 16183 raft_consensus.cc:2463] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 0.
W20250112 20:57:04.570021 16075 leader_election.cc:343] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.570104 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:04.569456 16298 raft_consensus.cc:383] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.570904 16306 leader_election.cc:290] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:04.570986 16075 leader_election.cc:304] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.571188 16298 raft_consensus.cc:738] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
W20250112 20:57:04.571884 16075 leader_election.cc:343] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.572180 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:04.571966 16298 consensus_queue.cc:260] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.572541 16305 raft_consensus.cc:2798] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:04.572737 16182 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.573051 16305 raft_consensus.cc:491] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:04.573410 16305 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.574585 16298 ts_tablet_manager.cc:1428] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20250112 20:57:04.575428 16298 tablet_bootstrap.cc:492] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.581969 16305 raft_consensus.cc:513] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.583576 16298 tablet_bootstrap.cc:654] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.584728 16305 leader_election.cc:290] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.584901 16297 tablet_bootstrap.cc:492] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.584741 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:04.585520 16297 ts_tablet_manager.cc:1397] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.137s user 0.011s sys 0.012s
I20250112 20:57:04.585479 16183 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.588836 16297 raft_consensus.cc:357] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.589753 16297 raft_consensus.cc:383] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.590118 16297 raft_consensus.cc:738] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.592010 16183 raft_consensus.cc:2463] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 1.
I20250112 20:57:04.594565 16302 raft_consensus.cc:491] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.594991 16302 raft_consensus.cc:513] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.596750 16302 leader_election.cc:290] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.603375 16182 raft_consensus.cc:2463] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 1.
I20250112 20:57:04.608372 16224 leader_election.cc:304] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.609455 16306 raft_consensus.cc:2798] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:04.591132 16297 consensus_queue.cc:260] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.622530 16298 tablet_bootstrap.cc:492] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.622969 16298 ts_tablet_manager.cc:1397] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.048s user 0.009s sys 0.005s
I20250112 20:57:04.623011 16075 leader_election.cc:304] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.624154 16306 raft_consensus.cc:695] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Becoming Leader. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:04.625069 16306 consensus_queue.cc:237] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.625751 16298 raft_consensus.cc:357] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.626657 16298 raft_consensus.cc:383] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.626938 16298 raft_consensus.cc:738] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.627665 16298 consensus_queue.cc:260] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.630374 16298 ts_tablet_manager.cc:1428] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20250112 20:57:04.631242 16298 tablet_bootstrap.cc:492] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.635645 16305 raft_consensus.cc:491] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.636054 16305 raft_consensus.cc:513] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.637934 16305 leader_election.cc:290] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.638468 16301 tablet_bootstrap.cc:654] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.638248 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.639413 16320 raft_consensus.cc:2798] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:04.640062 16257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.640472 16320 raft_consensus.cc:695] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 1 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:04.641443 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.641887 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.642517 16109 raft_consensus.cc:2463] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 0.
I20250112 20:57:04.642483 16320 consensus_queue.cc:237] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.643891 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:04.644440 16109 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:57:04.645685 16150 leader_election.cc:343] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:04.646322 16150 leader_election.cc:343] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.647570 16298 tablet_bootstrap.cc:654] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.649443 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:04.650250 16182 raft_consensus.cc:2388] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e59a26136604032abe0758f79a8562a in current term 1: Already voted for candidate 2f592694e8d94b6b9336474b40fa220c in this term.
I20250112 20:57:04.651728 16109 raft_consensus.cc:2463] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 1.
I20250112 20:57:04.652042 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.653800 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.655745 16297 ts_tablet_manager.cc:1428] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.070s user 0.007s sys 0.000s
I20250112 20:57:04.656172 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.656687 16297 tablet_bootstrap.cc:492] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.656486 16107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.656796 16108 raft_consensus.cc:2463] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 0.
I20250112 20:57:04.657236 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.658535 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:04.659868 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.660145 16298 tablet_bootstrap.cc:492] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.660570 16298 ts_tablet_manager.cc:1397] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.030s user 0.014s sys 0.001s
I20250112 20:57:04.661630 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.662904 16109 raft_consensus.cc:2388] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d5fc77120eda4e978f02d6c6fe3c08a8 in current term 1: Already voted for candidate 2f592694e8d94b6b9336474b40fa220c in this term.
I20250112 20:57:04.663347 16297 tablet_bootstrap.cc:654] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:04.656518 16075 leader_election.cc:343] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.662860 16298 raft_consensus.cc:357] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.663805 16298 raft_consensus.cc:383] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.663692 16075 leader_election.cc:304] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e59a26136604032abe0758f79a8562a; no voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:04.664180 16298 raft_consensus.cc:738] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
W20250112 20:57:04.664994 16075 leader_election.cc:343] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:04.665899 16075 leader_election.cc:343] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.665412 16005 catalog_manager.cc:5526] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f592694e8d94b6b9336474b40fa220c (127.15.23.131). New cstate: current_term: 1 leader_uuid: "2f592694e8d94b6b9336474b40fa220c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:04.665150 16298 consensus_queue.cc:260] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.669317 16298 ts_tablet_manager.cc:1428] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20250112 20:57:04.670214 16305 raft_consensus.cc:2743] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:04.670323 16298 tablet_bootstrap.cc:492] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.674620 16301 tablet_bootstrap.cc:492] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.675076 16301 ts_tablet_manager.cc:1397] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.236s user 0.015s sys 0.000s
I20250112 20:57:04.666311 16075 leader_election.cc:304] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e59a26136604032abe0758f79a8562a; no voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8
W20250112 20:57:04.677318 16075 leader_election.cc:343] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.677470 16298 tablet_bootstrap.cc:654] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.678246 16075 leader_election.cc:304] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e59a26136604032abe0758f79a8562a; no voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:04.680964 16328 raft_consensus.cc:2743] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:04.681597 16328 raft_consensus.cc:2743] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:04.682538 16147 leader_election.cc:343] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.682991 16147 leader_election.cc:304] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5fc77120eda4e978f02d6c6fe3c08a8; no voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a
I20250112 20:57:04.684020 16302 raft_consensus.cc:2743] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:04.685846 16147 leader_election.cc:304] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters: 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:04.686416 16302 raft_consensus.cc:2798] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:04.687014 16302 raft_consensus.cc:491] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:04.688864 16302 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.688984 16147 leader_election.cc:304] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5fc77120eda4e978f02d6c6fe3c08a8; no voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a
I20250112 20:57:04.693228 16301 raft_consensus.cc:357] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.694634 16335 raft_consensus.cc:2743] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:04.694903 16301 raft_consensus.cc:383] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.695343 16301 raft_consensus.cc:738] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.696183 16301 consensus_queue.cc:260] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.698338 16306 raft_consensus.cc:491] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.699309 16301 ts_tablet_manager.cc:1428] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.024s user 0.009s sys 0.000s
I20250112 20:57:04.698982 16306 raft_consensus.cc:513] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.699920 16301 tablet_bootstrap.cc:492] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.702047 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.702710 16109 raft_consensus.cc:2463] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 0.
I20250112 20:57:04.703253 16306 leader_election.cc:290] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:04.704097 16221 leader_election.cc:304] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:04.702617 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:04.706025 16298 tablet_bootstrap.cc:492] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.706575 16298 ts_tablet_manager.cc:1397] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.004s
I20250112 20:57:04.707563 16301 tablet_bootstrap.cc:654] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.698312 16302 raft_consensus.cc:513] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
W20250112 20:57:04.705513 16224 leader_election.cc:343] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.708004 16335 raft_consensus.cc:491] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.708329 16306 raft_consensus.cc:2798] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:04.709841 16306 raft_consensus.cc:491] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:04.709385 16335 raft_consensus.cc:513] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.710228 16306 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.712021 16335 leader_election.cc:290] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.712872 16297 tablet_bootstrap.cc:492] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.712944 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:04.713373 16297 ts_tablet_manager.cc:1397] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.011s
I20250112 20:57:04.713618 16109 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.714196 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.713073 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.718269 16298 raft_consensus.cc:357] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.718946 16298 raft_consensus.cc:383] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.719247 16298 raft_consensus.cc:738] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.719833 16298 consensus_queue.cc:260] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
W20250112 20:57:04.720834 16150 leader_election.cc:343] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.722625 16298 ts_tablet_manager.cc:1428] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20250112 20:57:04.723383 16298 tablet_bootstrap.cc:492] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8: Bootstrap starting.
I20250112 20:57:04.725920 16006 catalog_manager.cc:5526] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 1 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:04.728539 16302 leader_election.cc:290] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.728860 16297 raft_consensus.cc:357] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.731599 16297 raft_consensus.cc:383] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.731900 16297 raft_consensus.cc:738] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.732626 16297 consensus_queue.cc:260] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.733530 16108 raft_consensus.cc:2463] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 0.
I20250112 20:57:04.733165 16109 raft_consensus.cc:2463] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 1.
I20250112 20:57:04.729988 16306 raft_consensus.cc:513] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.737257 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:04.737800 16109 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.740121 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:04.740427 16298 tablet_bootstrap.cc:654] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.740823 16147 leader_election.cc:304] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters: 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:04.743072 16302 raft_consensus.cc:2798] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:04.743408 16302 raft_consensus.cc:491] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:04.743467 16147 leader_election.cc:304] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.743656 16109 raft_consensus.cc:2463] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 1.
I20250112 20:57:04.743804 16302 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.744305 16335 raft_consensus.cc:2798] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:04.744685 16221 leader_election.cc:304] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:04.745626 16326 raft_consensus.cc:2798] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:04.746690 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
W20250112 20:57:04.747726 16224 leader_election.cc:343] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.748255 16306 leader_election.cc:290] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
W20250112 20:57:04.749084 16150 leader_election.cc:343] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.749732 16297 ts_tablet_manager.cc:1428] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.036s user 0.006s sys 0.001s
I20250112 20:57:04.750568 16302 raft_consensus.cc:513] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.751539 16297 tablet_bootstrap.cc:492] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.752498 16302 leader_election.cc:290] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.753800 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:04.757071 16326 raft_consensus.cc:695] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Becoming Leader. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
W20250112 20:57:04.757854 16150 leader_election.cc:343] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.759732 16297 tablet_bootstrap.cc:654] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.754700 16301 tablet_bootstrap.cc:492] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.760512 16326 consensus_queue.cc:237] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.761418 16301 ts_tablet_manager.cc:1397] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.062s user 0.017s sys 0.014s
I20250112 20:57:04.764474 16301 raft_consensus.cc:357] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.765527 16301 raft_consensus.cc:383] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.765815 16301 raft_consensus.cc:738] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.766682 16301 consensus_queue.cc:260] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.767244 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:04.767815 16109 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.769094 16301 ts_tablet_manager.cc:1428] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250112 20:57:04.769968 16301 tablet_bootstrap.cc:492] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.771512 16006 catalog_manager.cc:5526] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f592694e8d94b6b9336474b40fa220c (127.15.23.131). New cstate: current_term: 1 leader_uuid: "2f592694e8d94b6b9336474b40fa220c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:04.776619 16335 raft_consensus.cc:695] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:04.777633 16301 tablet_bootstrap.cc:654] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.777503 16335 consensus_queue.cc:237] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.784420 16109 raft_consensus.cc:2463] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 1.
I20250112 20:57:04.785670 16147 leader_election.cc:304] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters: 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:04.785974 16298 tablet_bootstrap.cc:492] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8: No bootstrap required, opened a new log
I20250112 20:57:04.786520 16298 ts_tablet_manager.cc:1397] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent bootstrapping tablet: real 0.063s user 0.018s sys 0.012s
I20250112 20:57:04.787209 16340 raft_consensus.cc:2798] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:04.788409 16302 raft_consensus.cc:491] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.788856 16302 raft_consensus.cc:513] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.790822 16302 leader_election.cc:290] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.792378 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.792734 16340 raft_consensus.cc:695] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:04.793222 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.793920 16297 tablet_bootstrap.cc:492] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.793547 16340 consensus_queue.cc:237] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.793953 16301 tablet_bootstrap.cc:492] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
W20250112 20:57:04.794282 16150 leader_election.cc:343] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.794512 16297 ts_tablet_manager.cc:1397] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.010s
I20250112 20:57:04.794663 16301 ts_tablet_manager.cc:1397] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.008s
I20250112 20:57:04.793943 16108 raft_consensus.cc:2463] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 0.
I20250112 20:57:04.796437 16147 leader_election.cc:304] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters: 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:04.797183 16297 raft_consensus.cc:357] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.797410 16301 raft_consensus.cc:357] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.797863 16297 raft_consensus.cc:383] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.798151 16301 raft_consensus.cc:383] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.798206 16297 raft_consensus.cc:738] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.798555 16301 raft_consensus.cc:738] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.798456 16298 raft_consensus.cc:357] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.798974 16297 consensus_queue.cc:260] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.799494 16298 raft_consensus.cc:383] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.799932 16335 raft_consensus.cc:2798] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:04.800079 16298 raft_consensus.cc:738] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.800369 16335 raft_consensus.cc:491] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:04.799494 16301 consensus_queue.cc:260] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.800729 16335 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.800882 16298 consensus_queue.cc:260] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.803740 16301 ts_tablet_manager.cc:1428] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.009s user 0.004s sys 0.002s
I20250112 20:57:04.804843 16301 tablet_bootstrap.cc:492] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.803819 16298 ts_tablet_manager.cc:1428] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20250112 20:57:04.798630 16006 catalog_manager.cc:5526] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:04.809159 16335 raft_consensus.cc:513] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.811455 16301 tablet_bootstrap.cc:654] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.811781 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:04.812343 16109 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.814795 16297 ts_tablet_manager.cc:1428] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20250112 20:57:04.815686 16297 tablet_bootstrap.cc:492] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.821686 16297 tablet_bootstrap.cc:654] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.824652 16109 raft_consensus.cc:2463] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 1.
I20250112 20:57:04.826013 16147 leader_election.cc:304] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.827142 16340 raft_consensus.cc:2798] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:04.827684 16340 raft_consensus.cc:695] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:04.828671 16340 consensus_queue.cc:237] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.836840 16335 leader_election.cc:290] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.837850 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
W20250112 20:57:04.839216 16150 leader_election.cc:343] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.840579 16297 tablet_bootstrap.cc:492] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.841043 16297 ts_tablet_manager.cc:1397] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.016s
I20250112 20:57:04.842425 16301 tablet_bootstrap.cc:492] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.842844 16301 ts_tablet_manager.cc:1397] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.038s user 0.017s sys 0.014s
I20250112 20:57:04.843436 16297 raft_consensus.cc:357] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.843540 16006 catalog_manager.cc:5526] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:04.844126 16297 raft_consensus.cc:383] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.844401 16297 raft_consensus.cc:738] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.845002 16006 catalog_manager.cc:5526] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:04.845016 16297 consensus_queue.cc:260] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.845525 16301 raft_consensus.cc:357] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.846231 16301 raft_consensus.cc:383] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.846535 16301 raft_consensus.cc:738] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.847335 16297 ts_tablet_manager.cc:1428] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:57:04.847213 16301 consensus_queue.cc:260] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.849414 16297 tablet_bootstrap.cc:492] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.855731 16297 tablet_bootstrap.cc:654] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.856120 16301 ts_tablet_manager.cc:1428] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.013s user 0.007s sys 0.001s
I20250112 20:57:04.858417 16301 tablet_bootstrap.cc:492] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.863196 16297 tablet_bootstrap.cc:492] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.863638 16297 ts_tablet_manager.cc:1397] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20250112 20:57:04.864877 16301 tablet_bootstrap.cc:654] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.866132 16297 raft_consensus.cc:357] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.866791 16297 raft_consensus.cc:383] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.867063 16297 raft_consensus.cc:738] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.867774 16297 consensus_queue.cc:260] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.869660 16297 ts_tablet_manager.cc:1428] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250112 20:57:04.869978 16301 tablet_bootstrap.cc:492] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.870380 16301 ts_tablet_manager.cc:1397] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20250112 20:57:04.870528 16297 tablet_bootstrap.cc:492] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a: Bootstrap starting.
I20250112 20:57:04.872854 16301 raft_consensus.cc:357] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.873580 16301 raft_consensus.cc:383] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.873863 16301 raft_consensus.cc:738] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.874521 16301 consensus_queue.cc:260] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.875876 16297 tablet_bootstrap.cc:654] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.876628 16301 ts_tablet_manager.cc:1428] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:57:04.877562 16301 tablet_bootstrap.cc:492] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.881502 16297 tablet_bootstrap.cc:492] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a: No bootstrap required, opened a new log
I20250112 20:57:04.881897 16297 ts_tablet_manager.cc:1397] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20250112 20:57:04.883395 16301 tablet_bootstrap.cc:654] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.884104 16297 raft_consensus.cc:357] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.884816 16297 raft_consensus.cc:383] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.885134 16297 raft_consensus.cc:738] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.885727 16297 consensus_queue.cc:260] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.886981 16335 raft_consensus.cc:491] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.887408 16335 raft_consensus.cc:513] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.888311 16297 ts_tablet_manager.cc:1428] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250112 20:57:04.889250 16335 leader_election.cc:290] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.890727 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb464103f740429396ec85635d9853ca" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.891448 16256 raft_consensus.cc:2463] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 0.
I20250112 20:57:04.891919 16301 tablet_bootstrap.cc:492] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.890764 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb464103f740429396ec85635d9853ca" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.892439 16150 leader_election.cc:304] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.892642 16109 raft_consensus.cc:2463] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 0.
I20250112 20:57:04.893348 16301 ts_tablet_manager.cc:1397] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20250112 20:57:04.893291 16335 raft_consensus.cc:2798] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:04.893705 16335 raft_consensus.cc:491] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:04.894071 16335 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.899027 16335 raft_consensus.cc:513] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.900924 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb464103f740429396ec85635d9853ca" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:04.901104 16335 leader_election.cc:290] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.901407 16109 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.901317 16256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb464103f740429396ec85635d9853ca" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:04.901870 16256 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.901957 16335 raft_consensus.cc:491] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.902372 16335 raft_consensus.cc:513] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.902797 16301 raft_consensus.cc:357] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.903437 16301 raft_consensus.cc:383] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.903689 16301 raft_consensus.cc:738] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.903967 16340 raft_consensus.cc:491] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:04.904343 16340 raft_consensus.cc:513] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.905725 16335 leader_election.cc:290] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.905987 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.906180 16340 leader_election.cc:290] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.904318 16301 consensus_queue.cc:260] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.906704 16108 raft_consensus.cc:2388] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d5fc77120eda4e978f02d6c6fe3c08a8 in current term 1: Already voted for candidate 2f592694e8d94b6b9336474b40fa220c in this term.
I20250112 20:57:04.907721 16109 raft_consensus.cc:2463] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 1.
I20250112 20:57:04.908044 16257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:04.908581 16301 ts_tablet_manager.cc:1428] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250112 20:57:04.908525 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:04.909160 16108 raft_consensus.cc:2463] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 0.
I20250112 20:57:04.909576 16150 leader_election.cc:304] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5fc77120eda4e978f02d6c6fe3c08a8; no voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a
I20250112 20:57:04.909992 16147 leader_election.cc:304] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.910614 16340 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.910666 16335 raft_consensus.cc:2798] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:04.910712 16147 leader_election.cc:304] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:04.911608 16302 raft_consensus.cc:2798] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:04.911836 16335 raft_consensus.cc:695] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:04.911940 16302 raft_consensus.cc:491] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:04.912314 16302 raft_consensus.cc:3054] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.912341 16256 raft_consensus.cc:2463] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 1.
I20250112 20:57:04.913295 16301 tablet_bootstrap.cc:492] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.913816 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
W20250112 20:57:04.915010 16150 leader_election.cc:343] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.916450 16340 raft_consensus.cc:2743] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:04.916218 16335 consensus_queue.cc:237] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.917871 16302 raft_consensus.cc:513] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.919530 16301 tablet_bootstrap.cc:654] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.920746 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:04.921429 16108 raft_consensus.cc:3054] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:04.924052 16301 tablet_bootstrap.cc:492] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.924360 16301 ts_tablet_manager.cc:1397] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20250112 20:57:04.926332 16301 raft_consensus.cc:357] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.926812 16301 raft_consensus.cc:383] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.927004 16301 raft_consensus.cc:738] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.927465 16301 consensus_queue.cc:260] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.928898 16301 ts_tablet_manager.cc:1428] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20250112 20:57:04.929741 16301 tablet_bootstrap.cc:492] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.934796 16301 tablet_bootstrap.cc:654] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.939926 16302 leader_election.cc:290] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:04.940935 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:04.941358 16108 raft_consensus.cc:2463] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 1.
W20250112 20:57:04.942245 16150 leader_election.cc:343] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:04.943288 16147 leader_election.cc:304] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters: 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:04.944449 16335 raft_consensus.cc:2798] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:04.945010 16301 tablet_bootstrap.cc:492] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.945188 16335 raft_consensus.cc:695] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:04.945453 16301 ts_tablet_manager.cc:1397] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20250112 20:57:04.945919 16335 consensus_queue.cc:237] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.947785 16301 raft_consensus.cc:357] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.948503 16301 raft_consensus.cc:383] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.948817 16301 raft_consensus.cc:738] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.948875 16004 catalog_manager.cc:5526] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:04.949620 16301 consensus_queue.cc:260] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.951630 16301 ts_tablet_manager.cc:1428] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:57:04.952459 16301 tablet_bootstrap.cc:492] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c: Bootstrap starting.
I20250112 20:57:04.958690 16301 tablet_bootstrap.cc:654] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:04.963074 16006 catalog_manager.cc:5526] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:04.964869 16301 tablet_bootstrap.cc:492] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c: No bootstrap required, opened a new log
I20250112 20:57:04.965198 16301 ts_tablet_manager.cc:1397] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20250112 20:57:04.967454 16301 raft_consensus.cc:357] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.968016 16301 raft_consensus.cc:383] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:04.968207 16301 raft_consensus.cc:738] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Initialized, Role: FOLLOWER
I20250112 20:57:04.968729 16301 consensus_queue.cc:260] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:04.970372 16301 ts_tablet_manager.cc:1428] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20250112 20:57:05.015403 16326 consensus_queue.cc:1035] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.026968 16341 consensus_queue.cc:1035] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.034417 16335 raft_consensus.cc:491] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:05.034909 16335 raft_consensus.cc:513] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.036679 16335 leader_election.cc:290] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:05.037626 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac4a46114fc47ceb7f96990ab3bfa33" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c" is_pre_election: true
I20250112 20:57:05.037524 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac4a46114fc47ceb7f96990ab3bfa33" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:05.038177 16258 raft_consensus.cc:2463] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 0.
I20250112 20:57:05.038331 16108 raft_consensus.cc:2463] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 0.
I20250112 20:57:05.039124 16150 leader_election.cc:304] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:05.039834 16335 raft_consensus.cc:2798] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:05.040103 16335 raft_consensus.cc:491] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:05.040350 16335 raft_consensus.cc:3054] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.044278 16335 raft_consensus.cc:513] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.045873 16335 leader_election.cc:290] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:05.046435 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac4a46114fc47ceb7f96990ab3bfa33" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:05.046783 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac4a46114fc47ceb7f96990ab3bfa33" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:05.046975 16108 raft_consensus.cc:3054] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.047312 16258 raft_consensus.cc:3054] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.053056 16258 raft_consensus.cc:2463] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 1.
I20250112 20:57:05.053056 16108 raft_consensus.cc:2463] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 1.
I20250112 20:57:05.054023 16150 leader_election.cc:304] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:05.054737 16335 raft_consensus.cc:2798] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:05.055126 16335 raft_consensus.cc:695] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:05.055830 16335 consensus_queue.cc:237] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.063135 16258 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.062561 16004 catalog_manager.cc:5526] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:05.069007 16328 consensus_queue.cc:1035] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.078070 16305 consensus_queue.cc:1035] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.112299 16341 raft_consensus.cc:491] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:05.112715 16341 raft_consensus.cc:513] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.114132 16341 leader_election.cc:290] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:05.114746 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:05.114971 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:05.115324 16108 raft_consensus.cc:2388] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2f592694e8d94b6b9336474b40fa220c in current term 1: Already voted for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in this term.
I20250112 20:57:05.116098 16221 leader_election.cc:304] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c; no voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:05.116664 16341 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.120810 16341 raft_consensus.cc:2743] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:05.177891 16335 consensus_queue.cc:1035] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.185544 16258 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.192246 16341 raft_consensus.cc:491] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:05.192687 16341 raft_consensus.cc:513] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.192966 16335 consensus_queue.cc:1035] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.196095 16341 leader_election.cc:290] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:05.196311 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:05.196710 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:05.198148 16224 leader_election.cc:304] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c; no voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:05.199093 16341 raft_consensus.cc:2743] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:57:05.212390 16341 consensus_queue.cc:1035] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.213603 16341 raft_consensus.cc:491] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:05.214000 16341 raft_consensus.cc:513] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.215899 16341 leader_election.cc:290] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:05.216667 16108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:05.217221 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:05.217417 16108 raft_consensus.cc:2388] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2f592694e8d94b6b9336474b40fa220c in current term 1: Already voted for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in this term.
I20250112 20:57:05.218626 16224 leader_election.cc:304] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c; no voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:05.219349 16341 raft_consensus.cc:3054] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.222615 16326 raft_consensus.cc:491] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:05.223052 16326 raft_consensus.cc:513] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.224072 16258 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.225019 16326 leader_election.cc:290] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:05.226112 16341 raft_consensus.cc:2743] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:05.225971 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:05.226339 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
I20250112 20:57:05.226650 16109 raft_consensus.cc:2463] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 0.
I20250112 20:57:05.226873 16183 raft_consensus.cc:2463] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 0.
I20250112 20:57:05.227586 16221 leader_election.cc:304] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:05.229307 16341 raft_consensus.cc:2798] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:05.229638 16341 raft_consensus.cc:491] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:05.229951 16341 raft_consensus.cc:3054] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.230475 16306 consensus_queue.cc:1035] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.236428 16341 raft_consensus.cc:513] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.237072 16335 consensus_queue.cc:1035] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.238230 16341 leader_election.cc:290] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:05.238962 16109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:05.239430 16109 raft_consensus.cc:3054] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.245028 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:05.245649 16182 raft_consensus.cc:3054] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:05.247742 16109 raft_consensus.cc:2463] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 1.
I20250112 20:57:05.248651 16221 leader_election.cc:304] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:05.249289 16341 raft_consensus.cc:2798] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:05.249720 16341 raft_consensus.cc:695] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Becoming Leader. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:05.250519 16341 consensus_queue.cc:237] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.252046 16182 raft_consensus.cc:2463] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 1.
I20250112 20:57:05.255975 16335 consensus_queue.cc:1035] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:05.258052 16038 debug-util.cc:398] Leaking SignalData structure 0x7b0800053ba0 after lost signal to thread 15457
W20250112 20:57:05.258930 16038 debug-util.cc:398] Leaking SignalData structure 0x7b080004e280 after lost signal to thread 16041
I20250112 20:57:05.268357 16006 catalog_manager.cc:5526] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f592694e8d94b6b9336474b40fa220c (127.15.23.131). New cstate: current_term: 1 leader_uuid: "2f592694e8d94b6b9336474b40fa220c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:05.329479 16335 consensus_queue.cc:1035] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.337889 16335 consensus_queue.cc:1035] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:05.340981 16347 consensus_queue.cc:1035] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:05.350008 16347 consensus_queue.cc:1035] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:05.463248 16205 debug-util.cc:398] Leaking SignalData structure 0x7b08001040a0 after lost signal to thread 15457
I20250112 20:57:05.462996 16302 consensus_queue.cc:1035] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:05.464574 16205 debug-util.cc:398] Leaking SignalData structure 0x7b0800104780 after lost signal to thread 16041
W20250112 20:57:05.465658 16205 debug-util.cc:398] Leaking SignalData structure 0x7b08000e1e20 after lost signal to thread 16133
W20250112 20:57:05.465929 16205 debug-util.cc:398] Leaking SignalData structure 0x7b08001000e0 after lost signal to thread 16134
W20250112 20:57:05.466801 16205 debug-util.cc:398] Leaking SignalData structure 0x7b080009df00 after lost signal to thread 16209
I20250112 20:57:05.468376 16302 consensus_queue.cc:1035] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.484444 16302 consensus_queue.cc:1035] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.511691 16256 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d969c6c299574510b416acf29125bbfc"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:51784
I20250112 20:57:05.512395 16256 raft_consensus.cc:604] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:05.518054 16257 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:51784
I20250112 20:57:05.518612 16257 raft_consensus.cc:604] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:05.533113 16340 consensus_queue.cc:1035] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.576622 16183 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:60712
I20250112 20:57:05.577210 16183 raft_consensus.cc:604] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:05.582340 16258 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:51784
I20250112 20:57:05.582796 16258 raft_consensus.cc:604] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:05.586433 16183 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:60712
I20250112 20:57:05.586818 16183 raft_consensus.cc:604] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:05.606490 16108 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
mode: GRACEFUL
new_leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
from {username='slave'} at 127.0.0.1:50820
I20250112 20:57:05.606937 16108 raft_consensus.cc:604] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 1 LEADER]: Received request to transfer leadership to TS d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:05.616068 16183 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
mode: GRACEFUL
new_leader_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:60712
I20250112 20:57:05.616511 16183 raft_consensus.cc:604] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Received request to transfer leadership to TS 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:05.616946 16306 raft_consensus.cc:988] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.617331 16306 raft_consensus.cc:1076] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.618847 16108 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35052
I20250112 20:57:05.619369 16108 raft_consensus.cc:491] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:05.619685 16108 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.627763 16108 raft_consensus.cc:513] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.629155 16108 leader_election.cc:290] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:05.629880 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:05.630451 16183 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.636229 16183 raft_consensus.cc:2463] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.637756 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:05.638190 16258 raft_consensus.cc:3049] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:05.638434 16258 raft_consensus.cc:738] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:05.638988 16258 consensus_queue.cc:260] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.639827 16258 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.643541 16258 raft_consensus.cc:2463] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.637384 16075 leader_election.cc:304] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:05.644524 16320 raft_consensus.cc:2798] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:05.645624 16320 raft_consensus.cc:695] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:05.646391 16320 consensus_queue.cc:237] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.652022 16341 raft_consensus.cc:988] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.652360 16372 raft_consensus.cc:1076] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.653951 16006 catalog_manager.cc:5526] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 1 to 2, leader changed from 2f592694e8d94b6b9336474b40fa220c (127.15.23.131) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 2 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:05.656505 16108 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d969c6c299574510b416acf29125bbfc"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35052
I20250112 20:57:05.657086 16108 raft_consensus.cc:491] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:05.657404 16108 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.663280 16108 raft_consensus.cc:513] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.672299 16183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:05.673648 16183 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.674639 16341 consensus_queue.cc:1035] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:05.675629 16108 leader_election.cc:290] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:05.689929 16390 raft_consensus.cc:988] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a: : Instructing follower d5fc77120eda4e978f02d6c6fe3c08a8 to start an election
I20250112 20:57:05.691208 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:05.697820 16335 raft_consensus.cc:988] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.698698 16335 raft_consensus.cc:1076] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.699972 16258 raft_consensus.cc:3049] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:05.700464 16258 raft_consensus.cc:738] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:05.702889 16108 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35064
I20250112 20:57:05.702186 16258 consensus_queue.cc:260] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.705425 16258 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.712278 16341 consensus_queue.cc:1035] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:05.704322 16108 raft_consensus.cc:491] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:05.719305 16108 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.725041 16391 raft_consensus.cc:1076] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 1 LEADER]: Signalling peer d5fc77120eda4e978f02d6c6fe3c08a8 to start an election
I20250112 20:57:05.726717 16184 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
from {username='slave'} at 127.0.0.1:54942
I20250112 20:57:05.727309 16184 raft_consensus.cc:491] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:05.727660 16184 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.733919 16183 raft_consensus.cc:2463] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.734483 16184 raft_consensus.cc:513] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.751329 16075 leader_election.cc:304] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:05.751588 16184 leader_election.cc:290] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 2 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:05.752859 16391 raft_consensus.cc:2798] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:05.753307 16391 raft_consensus.cc:695] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:05.754006 16391 consensus_queue.cc:237] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.756494 16257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:05.757067 16257 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.757434 16107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:05.757951 16107 raft_consensus.cc:3049] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:05.758220 16107 raft_consensus.cc:738] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:05.758857 16107 consensus_queue.cc:260] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.760077 16107 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.766966 16302 raft_consensus.cc:988] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.767366 16302 raft_consensus.cc:1076] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.771795 16106 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35064
I20250112 20:57:05.772289 16106 raft_consensus.cc:491] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:05.772629 16106 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.775692 16257 raft_consensus.cc:2463] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 2.
I20250112 20:57:05.776715 16150 leader_election.cc:304] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:05.777626 16347 raft_consensus.cc:2798] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:05.778108 16347 raft_consensus.cc:695] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:05.778682 16106 raft_consensus.cc:513] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.778923 16347 consensus_queue.cc:237] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.780579 16258 raft_consensus.cc:2463] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.782083 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:05.781523 16108 raft_consensus.cc:513] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.782711 16182 raft_consensus.cc:3049] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:05.783075 16182 raft_consensus.cc:738] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:05.784236 16108 leader_election.cc:290] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:05.784909 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:05.787725 16184 raft_consensus.cc:3049] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:05.787856 16257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:05.788069 16184 raft_consensus.cc:738] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:05.788434 16257 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.788802 16184 consensus_queue.cc:260] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.789902 16184 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.791841 16182 consensus_queue.cc:260] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.793015 16182 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.793816 16007 catalog_manager.cc:5526] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 1 to 2, leader changed from 6e59a26136604032abe0758f79a8562a (127.15.23.129) to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:05.796451 16341 raft_consensus.cc:988] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.797236 16341 raft_consensus.cc:1076] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:05.806298 16184 raft_consensus.cc:2463] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.806646 16182 raft_consensus.cc:2463] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.787164 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:05.807448 16258 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.808980 16107 raft_consensus.cc:2463] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 2.
I20250112 20:57:05.809526 16106 leader_election.cc:290] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:05.811327 16105 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35052
I20250112 20:57:05.820142 16105 raft_consensus.cc:491] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:05.820580 16105 raft_consensus.cc:3054] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.821576 16258 raft_consensus.cc:2463] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.794588 16257 raft_consensus.cc:2463] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.815265 16006 catalog_manager.cc:5526] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 1 to 2, leader changed from 2f592694e8d94b6b9336474b40fa220c (127.15.23.131) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 2 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:05.827553 16105 raft_consensus.cc:513] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.829593 16105 leader_election.cc:290] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:05.816715 16075 leader_election.cc:304] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:05.831780 16075 leader_election.cc:304] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:05.832571 16390 raft_consensus.cc:2798] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:05.833119 16390 raft_consensus.cc:695] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:05.833926 16390 consensus_queue.cc:237] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.835872 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:05.836427 16258 raft_consensus.cc:3049] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:05.836772 16258 raft_consensus.cc:738] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:05.837394 16389 raft_consensus.cc:2798] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:05.837574 16258 consensus_queue.cc:260] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.842741 16389 raft_consensus.cc:695] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:05.843050 16258 raft_consensus.cc:3054] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.843612 16389 consensus_queue.cc:237] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.838811 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51f70e21e6804833a0e56fd3436032cc" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:05.848023 16182 raft_consensus.cc:3054] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.851171 16258 raft_consensus.cc:2463] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.852206 16075 leader_election.cc:304] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:05.853024 16389 raft_consensus.cc:2798] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:05.853605 16389 raft_consensus.cc:695] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:05.854421 16389 consensus_queue.cc:237] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.858678 16006 catalog_manager.cc:5526] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 1 to 2, leader changed from d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 2 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:05.859793 16182 raft_consensus.cc:2463] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:05.875068 16006 catalog_manager.cc:5526] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 1 to 2, leader changed from d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 2 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:05.876696 16006 catalog_manager.cc:5526] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 1 to 2, leader changed from 2f592694e8d94b6b9336474b40fa220c (127.15.23.131) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 2 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:05.963420 16335 raft_consensus.cc:988] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8: : Instructing follower 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:05.963930 16347 raft_consensus.cc:1076] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Signalling peer 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:05.965631 16258 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:45570
I20250112 20:57:05.966207 16258 raft_consensus.cc:491] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:05.966585 16258 raft_consensus.cc:3054] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.974190 16258 raft_consensus.cc:513] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.977224 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:05.977326 16105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e32cc98919f4a5fad3816a08f4a4b18" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:05.977838 16182 raft_consensus.cc:3049] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:05.977967 16105 raft_consensus.cc:3054] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.978190 16182 raft_consensus.cc:738] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:05.979001 16182 consensus_queue.cc:260] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.980152 16182 raft_consensus.cc:3054] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:05.986199 16105 raft_consensus.cc:2463] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 2.
I20250112 20:57:05.987082 16182 raft_consensus.cc:2463] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 2.
I20250112 20:57:05.987444 16221 leader_election.cc:304] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:05.988288 16258 leader_election.cc:290] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 2 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:05.988335 16372 raft_consensus.cc:2798] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:05.989048 16372 raft_consensus.cc:695] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 2 LEADER]: Becoming Leader. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:05.989889 16372 consensus_queue.cc:237] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:05.997875 16006 catalog_manager.cc:5526] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c reported cstate change: term changed from 1 to 2, leader changed from d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130) to 2f592694e8d94b6b9336474b40fa220c (127.15.23.131). New cstate: current_term: 2 leader_uuid: "2f592694e8d94b6b9336474b40fa220c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:06.113776 16258 raft_consensus.cc:1270] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.115123 16390 consensus_queue.cc:1035] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.134860 16182 raft_consensus.cc:1270] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.136672 16390 consensus_queue.cc:1035] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.164286 16258 raft_consensus.cc:1270] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Refusing update from remote peer d5fc77120eda4e978f02d6c6fe3c08a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.165678 16347 consensus_queue.cc:1035] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.178052 16105 raft_consensus.cc:1270] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Refusing update from remote peer d5fc77120eda4e978f02d6c6fe3c08a8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.179531 16335 consensus_queue.cc:1035] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.229997 16184 raft_consensus.cc:1270] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.231480 16390 consensus_queue.cc:1035] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.241607 16182 raft_consensus.cc:1270] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.245143 16389 consensus_queue.cc:1035] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.246747 16258 raft_consensus.cc:1270] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.249991 16391 consensus_queue.cc:1035] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.265648 16257 raft_consensus.cc:1270] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.268781 16391 consensus_queue.cc:1035] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.277172 16182 raft_consensus.cc:1270] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.278661 16391 consensus_queue.cc:1035] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.289422 16256 raft_consensus.cc:1270] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.291260 16390 consensus_queue.cc:1035] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:06.334897 16257 raft_consensus.cc:1270] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.336689 16391 consensus_queue.cc:1035] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.352866 16184 raft_consensus.cc:1270] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.354641 16389 consensus_queue.cc:1035] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.503090 16105 raft_consensus.cc:1270] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.504648 16372 consensus_queue.cc:1035] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:06.534319 16184 raft_consensus.cc:1270] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:06.542650 16403 consensus_queue.cc:1035] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:08.649559 15454 auto_leader_rebalancer-test.cc:339] The leader distribution is
I20250112 20:57:08.650260 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:08.677095 16105 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
mode: GRACEFUL
new_leader_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:50830
I20250112 20:57:08.677739 16105 raft_consensus.cc:604] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Received request to transfer leadership to TS 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:08.680240 16105 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
mode: GRACEFUL
new_leader_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:50830
I20250112 20:57:08.680855 16105 raft_consensus.cc:604] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Received request to transfer leadership to TS 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:08.681869 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:57:08.682386 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:08.697283 16408 raft_consensus.cc:988] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a: : Instructing follower 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:08.697774 16390 raft_consensus.cc:1076] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Signalling peer 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:08.699165 16257 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:45560
I20250112 20:57:08.699717 16257 raft_consensus.cc:491] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:08.700109 16257 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:08.707407 16257 raft_consensus.cc:513] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:08.709471 16257 leader_election.cc:290] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:08.710126 16105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:08.710529 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:08.710742 16105 raft_consensus.cc:3049] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:57:08.711123 16105 raft_consensus.cc:738] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:08.711148 16184 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:08.711820 16105 consensus_queue.cc:260] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:08.712735 16105 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:08.716682 16105 raft_consensus.cc:2463] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 3.
I20250112 20:57:08.717553 16221 leader_election.cc:304] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:08.717859 16184 raft_consensus.cc:2463] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 3.
I20250112 20:57:08.718135 16372 raft_consensus.cc:2798] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:57:08.719357 16372 raft_consensus.cc:695] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Becoming Leader. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:08.720094 16372 consensus_queue.cc:237] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:08.727114 16004 catalog_manager.cc:5526] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c reported cstate change: term changed from 2 to 3, leader changed from 6e59a26136604032abe0758f79a8562a (127.15.23.129) to 2f592694e8d94b6b9336474b40fa220c (127.15.23.131). New cstate: current_term: 3 leader_uuid: "2f592694e8d94b6b9336474b40fa220c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:09.070609 16408 raft_consensus.cc:988] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a: : Instructing follower 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:09.071110 16390 raft_consensus.cc:1076] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Signalling peer 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:09.072513 16257 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:45560
I20250112 20:57:09.073107 16257 raft_consensus.cc:491] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:09.073463 16257 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:09.080812 16257 raft_consensus.cc:513] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:09.082780 16257 leader_election.cc:290] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:09.083367 16105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:09.083748 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:09.084384 16105 raft_consensus.cc:3049] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:57:09.084408 16184 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:09.084853 16105 raft_consensus.cc:738] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:09.085634 16105 consensus_queue.cc:260] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:09.086810 16105 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:09.090703 16184 raft_consensus.cc:2463] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 3.
I20250112 20:57:09.091686 16224 leader_election.cc:304] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:09.092798 16105 raft_consensus.cc:2463] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 3.
I20250112 20:57:09.093009 16409 raft_consensus.cc:2798] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:57:09.093530 16409 raft_consensus.cc:695] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Becoming Leader. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:09.094244 16409 consensus_queue.cc:237] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:09.103397 16004 catalog_manager.cc:5526] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c reported cstate change: term changed from 2 to 3, leader changed from 6e59a26136604032abe0758f79a8562a (127.15.23.129) to 2f592694e8d94b6b9336474b40fa220c (127.15.23.131). New cstate: current_term: 3 leader_uuid: "2f592694e8d94b6b9336474b40fa220c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:09.248266 16184 raft_consensus.cc:1270] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:09.249428 16409 consensus_queue.cc:1035] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:09.258785 16105 raft_consensus.cc:1270] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:09.262224 16418 consensus_queue.cc:1035] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:09.539700 16184 raft_consensus.cc:1270] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:09.541939 16418 consensus_queue.cc:1035] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:09.968994 16107 raft_consensus.cc:1270] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:10.132320 16372 consensus_queue.cc:1035] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:10.264051 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:10.269284 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:10.269657 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20250112 20:57:10.955781 16205 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5e60 after lost signal to thread 15457
W20250112 20:57:10.956444 16205 debug-util.cc:398] Leaking SignalData structure 0x7b0800140300 after lost signal to thread 16041
W20250112 20:57:10.957146 16205 debug-util.cc:398] Leaking SignalData structure 0x7b0800104420 after lost signal to thread 16133
W20250112 20:57:10.957759 16205 debug-util.cc:398] Leaking SignalData structure 0x7b08000e62e0 after lost signal to thread 16208
W20250112 20:57:10.958359 16205 debug-util.cc:398] Leaking SignalData structure 0x7b08001a47c0 after lost signal to thread 16282
I20250112 20:57:11.270366 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:11.276197 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:11.276654 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:12.277350 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:12.282351 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:12.282732 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:13.283720 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:13.288816 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:13.289186 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:14.289899 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:14.294723 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:14.295071 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:15.295789 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:15.300360 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:15.300735 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:16.301505 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:16.306190 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:16.306614 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:17.307337 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:17.312187 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:17.312551 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:18.313280 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:18.318101 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:18.318445 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:19.907461 16447 raft_consensus.cc:491] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6e59a26136604032abe0758f79a8562a)
I20250112 20:57:19.908046 16447 raft_consensus.cc:513] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:19.911336 16447 leader_election.cc:290] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:19.916486 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" is_pre_election: true
W20250112 20:57:19.918705 16074 thread.cc:635] raft [worker] (thread pool) Time spent creating pthread: real 0.717s user 0.009s sys 0.024s
W20250112 20:57:19.919055 16074 thread.cc:601] raft [worker] (thread pool) Time spent starting thread: real 0.718s user 0.009s sys 0.025s
I20250112 20:57:19.930030 16105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6e59a26136604032abe0758f79a8562a" is_pre_election: true
I20250112 20:57:19.932850 16221 leader_election.cc:304] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c; no voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:19.934132 16447 raft_consensus.cc:2743] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250112 20:57:19.959425 15454 auto_leader_rebalancer-test.cc:358] The leader distribution is
I20250112 20:57:19.966413 16257 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:51784
I20250112 20:57:19.967211 16257 raft_consensus.cc:604] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:19.971859 16257 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:51784
I20250112 20:57:19.972434 16257 raft_consensus.cc:604] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:19.981833 16182 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:60712
I20250112 20:57:19.982324 16182 raft_consensus.cc:604] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:19.986613 16182 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:60712
I20250112 20:57:19.987025 16182 raft_consensus.cc:604] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
I20250112 20:57:19.991186 16182 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cb464103f740429396ec85635d9853ca"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
mode: GRACEFUL
new_leader_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:60712
I20250112 20:57:19.991591 16182 raft_consensus.cc:604] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Received request to transfer leadership to TS 6e59a26136604032abe0758f79a8562a
W20250112 20:57:20.025846 16130 debug-util.cc:398] Leaking SignalData structure 0x7b08001a17e0 after lost signal to thread 15457
W20250112 20:57:20.026971 16130 debug-util.cc:398] Leaking SignalData structure 0x7b08001cfbe0 after lost signal to thread 16041
W20250112 20:57:20.028159 16130 debug-util.cc:398] Leaking SignalData structure 0x7b08000eb260 after lost signal to thread 16133
W20250112 20:57:20.029207 16130 debug-util.cc:398] Leaking SignalData structure 0x7b08000f1e40 after lost signal to thread 16208
W20250112 20:57:20.030234 16130 debug-util.cc:398] Leaking SignalData structure 0x7b08001041a0 after lost signal to thread 16282
I20250112 20:57:20.333840 16432 raft_consensus.cc:988] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.334508 16447 raft_consensus.cc:988] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.334683 16448 raft_consensus.cc:1076] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.335009 16447 raft_consensus.cc:1076] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.336911 16105 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35052
I20250112 20:57:20.336859 16106 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35052
I20250112 20:57:20.337489 16105 raft_consensus.cc:491] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:20.337595 16106 raft_consensus.cc:491] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:20.337814 16105 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:57:20.338047 16106 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:57:20.342993 16105 raft_consensus.cc:513] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.344902 16105 leader_election.cc:290] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 4 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:20.345594 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:20.346088 16182 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:57:20.345702 16106 raft_consensus.cc:513] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.348057 16257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85ad275e24e24a3eafa4192cc2e00929" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:20.348659 16257 raft_consensus.cc:3049] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Stepping down as leader of term 3
I20250112 20:57:20.348958 16257 raft_consensus.cc:738] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:20.349385 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:20.349879 16184 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:57:20.349840 16106 leader_election.cc:290] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 4 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:20.349692 16257 consensus_queue.cc:260] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.350973 16257 raft_consensus.cc:3054] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:57:20.351687 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986bdc5230e9412092af88b2f5d1fb5b" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:20.352218 16258 raft_consensus.cc:3049] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Stepping down as leader of term 3
I20250112 20:57:20.352485 16258 raft_consensus.cc:738] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:20.353247 16258 consensus_queue.cc:260] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.354545 16258 raft_consensus.cc:3054] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:57:20.356364 16184 raft_consensus.cc:2463] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 4.
I20250112 20:57:20.357867 16257 raft_consensus.cc:2463] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 4.
I20250112 20:57:20.361008 16182 raft_consensus.cc:2463] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 4.
I20250112 20:57:20.362249 16075 leader_election.cc:304] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:20.363128 16436 raft_consensus.cc:2798] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 4 FOLLOWER]: Leader election won for term 4
I20250112 20:57:20.363147 16075 leader_election.cc:304] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:20.363448 16258 raft_consensus.cc:2463] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 4.
I20250112 20:57:20.364177 16444 raft_consensus.cc:2798] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 4 FOLLOWER]: Leader election won for term 4
I20250112 20:57:20.364784 16444 raft_consensus.cc:695] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 4 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:20.365027 16436 raft_consensus.cc:695] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 4 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:20.366183 16436 consensus_queue.cc:237] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.365980 16444 consensus_queue.cc:237] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.378175 16004 catalog_manager.cc:5526] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 3 to 4, leader changed from 2f592694e8d94b6b9336474b40fa220c (127.15.23.131) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 4 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:20.396271 16004 catalog_manager.cc:5526] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 3 to 4, leader changed from 2f592694e8d94b6b9336474b40fa220c (127.15.23.131) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 4 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:20.452338 16437 raft_consensus.cc:988] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.452939 16457 raft_consensus.cc:1076] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.454212 16106 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb464103f740429396ec85635d9853ca"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35064
I20250112 20:57:20.454800 16106 raft_consensus.cc:491] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:20.455163 16106 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:20.462488 16106 raft_consensus.cc:513] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.464501 16106 leader_election.cc:290] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:20.465346 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb464103f740429396ec85635d9853ca" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:20.465240 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb464103f740429396ec85635d9853ca" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:20.465857 16258 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:20.466185 16182 raft_consensus.cc:3049] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:20.466527 16182 raft_consensus.cc:738] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:20.467267 16182 consensus_queue.cc:260] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.468521 16182 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:20.471830 16258 raft_consensus.cc:2463] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:20.472872 16457 raft_consensus.cc:988] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.472916 16075 leader_election.cc:304] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:20.473510 16457 raft_consensus.cc:1076] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.473877 16444 raft_consensus.cc:2798] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:20.474386 16444 raft_consensus.cc:695] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:20.475236 16444 consensus_queue.cc:237] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.475634 16182 raft_consensus.cc:2463] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:20.476521 16106 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35064
I20250112 20:57:20.477200 16106 raft_consensus.cc:491] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:20.478055 16106 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:20.483381 16106 raft_consensus.cc:513] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.483848 16004 catalog_manager.cc:5526] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 1 to 2, leader changed from d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 2 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:20.484800 16106 leader_election.cc:290] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:20.485666 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:20.486203 16182 raft_consensus.cc:3049] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:57:20.486527 16182 raft_consensus.cc:738] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:20.486577 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:20.487128 16258 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:20.487381 16182 consensus_queue.cc:260] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.488543 16182 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:57:20.493080 16258 raft_consensus.cc:2463] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:20.494524 16182 raft_consensus.cc:2463] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 2.
I20250112 20:57:20.494453 16075 leader_election.cc:304] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:20.495246 16444 raft_consensus.cc:2798] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:57:20.495663 16444 raft_consensus.cc:695] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:20.496344 16444 consensus_queue.cc:237] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.503944 16004 catalog_manager.cc:5526] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 1 to 2, leader changed from d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 2 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:20.521296 16437 raft_consensus.cc:988] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8: : Instructing follower 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.521679 16457 raft_consensus.cc:1076] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 LEADER]: Signalling peer 6e59a26136604032abe0758f79a8562a to start an election
I20250112 20:57:20.522907 16106 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
from {username='slave'} at 127.0.0.1:35064
I20250112 20:57:20.523821 16106 raft_consensus.cc:491] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:20.524093 16106 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:20.528260 16106 raft_consensus.cc:513] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.529580 16106 leader_election.cc:290] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 3 election: Requested vote from peers d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:20.530352 16182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:20.530516 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f78ae8287d494781c3d4b9fae23e2f" candidate_uuid: "6e59a26136604032abe0758f79a8562a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:20.530979 16182 raft_consensus.cc:3049] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:57:20.531014 16258 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:20.534798 16182 raft_consensus.cc:738] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:20.534798 16258 raft_consensus.cc:2463] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 3.
I20250112 20:57:20.535640 16182 consensus_queue.cc:260] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.536805 16075 leader_election.cc:304] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, 6e59a26136604032abe0758f79a8562a; no voters:
I20250112 20:57:20.536936 16182 raft_consensus.cc:3054] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:20.538094 16444 raft_consensus.cc:2798] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:57:20.538463 16444 raft_consensus.cc:695] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 3 LEADER]: Becoming Leader. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:20.539191 16444 consensus_queue.cc:237] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:20.543820 16182 raft_consensus.cc:2463] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e59a26136604032abe0758f79a8562a in term 3.
I20250112 20:57:20.546134 16004 catalog_manager.cc:5526] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a reported cstate change: term changed from 2 to 3, leader changed from d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130) to 6e59a26136604032abe0758f79a8562a (127.15.23.129). New cstate: current_term: 3 leader_uuid: "6e59a26136604032abe0758f79a8562a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:20.817481 16258 raft_consensus.cc:1270] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 4 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:57:20.818717 16444 consensus_queue.cc:1035] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:20.830986 16182 raft_consensus.cc:1270] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 4 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:57:20.832487 16436 consensus_queue.cc:1035] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:20.842654 16184 raft_consensus.cc:1270] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 4 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:57:20.844058 16436 consensus_queue.cc:1035] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:20.856523 16258 raft_consensus.cc:1270] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 4 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:57:20.859813 16450 consensus_queue.cc:1035] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:20.875507 16258 raft_consensus.cc:1270] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:20.877089 16450 consensus_queue.cc:1035] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:20.879565 16184 raft_consensus.cc:1270] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:20.880653 16444 consensus_queue.cc:1035] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:20.900374 16182 raft_consensus.cc:1270] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:20.901659 16436 consensus_queue.cc:1035] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:20.910878 16257 raft_consensus.cc:1270] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:57:20.912220 16450 consensus_queue.cc:1035] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:20.984468 16258 raft_consensus.cc:1270] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:20.985543 16450 consensus_queue.cc:1035] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:20.996687 16182 raft_consensus.cc:1270] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Refusing update from remote peer 6e59a26136604032abe0758f79a8562a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:20.998127 16473 consensus_queue.cc:1035] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250112 20:57:22.517552 16279 debug-util.cc:398] Leaking SignalData structure 0x7b08000ab5a0 after lost signal to thread 15457
W20250112 20:57:22.518234 16279 debug-util.cc:398] Leaking SignalData structure 0x7b08000abc80 after lost signal to thread 16041
W20250112 20:57:22.519016 16279 debug-util.cc:398] Leaking SignalData structure 0x7b0800197780 after lost signal to thread 16133
W20250112 20:57:22.519707 16279 debug-util.cc:398] Leaking SignalData structure 0x7b08000f9b20 after lost signal to thread 16208
W20250112 20:57:22.520385 16279 debug-util.cc:398] Leaking SignalData structure 0x7b080015a100 after lost signal to thread 16282
I20250112 20:57:23.100302 15454 auto_leader_rebalancer-test.cc:373] The leader distribution is
I20250112 20:57:23.101118 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:23.108126 16106 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
mode: GRACEFUL
new_leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
from {username='slave'} at 127.0.0.1:50830
I20250112 20:57:23.109169 16106 raft_consensus.cc:604] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Received request to transfer leadership to TS d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:23.111811 16106 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cb464103f740429396ec85635d9853ca"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
mode: GRACEFUL
new_leader_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:50830
I20250112 20:57:23.112370 16106 raft_consensus.cc:604] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Received request to transfer leadership to TS 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:23.114683 16106 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d969c6c299574510b416acf29125bbfc"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
mode: GRACEFUL
new_leader_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:50830
I20250112 20:57:23.115238 16106 raft_consensus.cc:604] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Received request to transfer leadership to TS 2f592694e8d94b6b9336474b40fa220c
I20250112 20:57:23.117470 16106 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203"
dest_uuid: "6e59a26136604032abe0758f79a8562a"
mode: GRACEFUL
new_leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
from {username='slave'} at 127.0.0.1:50830
I20250112 20:57:23.118044 16106 raft_consensus.cc:604] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Received request to transfer leadership to TS d5fc77120eda4e978f02d6c6fe3c08a8
I20250112 20:57:23.119649 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20250112 20:57:23.120288 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:23.150382 16476 raft_consensus.cc:988] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a: : Instructing follower 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:23.150930 16473 raft_consensus.cc:1076] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Signalling peer 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:23.152357 16257 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb464103f740429396ec85635d9853ca"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:45560
I20250112 20:57:23.152993 16257 raft_consensus.cc:491] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:23.153481 16257 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.162582 16257 raft_consensus.cc:513] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.164948 16257 leader_election.cc:290] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:23.165905 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb464103f740429396ec85635d9853ca" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:23.165791 16106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb464103f740429396ec85635d9853ca" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:23.166589 16184 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.166751 16106 raft_consensus.cc:3049] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:57:23.167093 16106 raft_consensus.cc:738] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:23.167953 16106 consensus_queue.cc:260] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.169340 16106 raft_consensus.cc:3054] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.174089 16184 raft_consensus.cc:2463] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 3.
I20250112 20:57:23.175634 16224 leader_election.cc:304] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:23.176366 16106 raft_consensus.cc:2463] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 3.
I20250112 20:57:23.176520 16449 raft_consensus.cc:2798] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:57:23.177796 16449 raft_consensus.cc:695] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Becoming Leader. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:23.178943 16449 consensus_queue.cc:237] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.189929 16007 catalog_manager.cc:5526] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c reported cstate change: term changed from 2 to 3, leader changed from 6e59a26136604032abe0758f79a8562a (127.15.23.129) to 2f592694e8d94b6b9336474b40fa220c (127.15.23.131). New cstate: current_term: 3 leader_uuid: "2f592694e8d94b6b9336474b40fa220c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:23.262277 16476 raft_consensus.cc:988] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a: : Instructing follower 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:23.262831 16473 raft_consensus.cc:1076] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Signalling peer 2f592694e8d94b6b9336474b40fa220c to start an election
I20250112 20:57:23.264061 16257 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d969c6c299574510b416acf29125bbfc"
dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
from {username='slave'} at 127.0.0.1:45560
I20250112 20:57:23.264685 16257 raft_consensus.cc:491] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:23.265058 16257 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.272449 16257 raft_consensus.cc:513] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.274359 16257 leader_election.cc:290] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130:36113)
I20250112 20:57:23.275254 16106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:23.275358 16184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d969c6c299574510b416acf29125bbfc" candidate_uuid: "2f592694e8d94b6b9336474b40fa220c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
I20250112 20:57:23.275884 16106 raft_consensus.cc:3049] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:57:23.275966 16184 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.276172 16106 raft_consensus.cc:738] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:23.276960 16106 consensus_queue.cc:260] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.278352 16106 raft_consensus.cc:3054] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.282428 16184 raft_consensus.cc:2463] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 3.
I20250112 20:57:23.283382 16224 leader_election.cc:304] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:23.284162 16449 raft_consensus.cc:2798] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:57:23.284319 16106 raft_consensus.cc:2463] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f592694e8d94b6b9336474b40fa220c in term 3.
I20250112 20:57:23.284690 16449 raft_consensus.cc:695] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Becoming Leader. State: Replica: 2f592694e8d94b6b9336474b40fa220c, State: Running, Role: LEADER
I20250112 20:57:23.285636 16449 consensus_queue.cc:237] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.293512 16007 catalog_manager.cc:5526] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c reported cstate change: term changed from 2 to 3, leader changed from 6e59a26136604032abe0758f79a8562a (127.15.23.129) to 2f592694e8d94b6b9336474b40fa220c (127.15.23.131). New cstate: current_term: 3 leader_uuid: "2f592694e8d94b6b9336474b40fa220c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:23.406273 16476 raft_consensus.cc:988] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a: : Instructing follower d5fc77120eda4e978f02d6c6fe3c08a8 to start an election
I20250112 20:57:23.406838 16473 raft_consensus.cc:1076] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Signalling peer d5fc77120eda4e978f02d6c6fe3c08a8 to start an election
I20250112 20:57:23.408133 16184 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
from {username='slave'} at 127.0.0.1:54942
I20250112 20:57:23.408686 16184 raft_consensus.cc:491] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:23.409036 16184 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.413929 16476 raft_consensus.cc:988] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a: : Instructing follower d5fc77120eda4e978f02d6c6fe3c08a8 to start an election
I20250112 20:57:23.414321 16473 raft_consensus.cc:1076] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Signalling peer d5fc77120eda4e978f02d6c6fe3c08a8 to start an election
I20250112 20:57:23.415489 16182 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203"
dest_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8"
from {username='slave'} at 127.0.0.1:54942
I20250112 20:57:23.415880 16182 raft_consensus.cc:491] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:57:23.415578 16184 raft_consensus.cc:513] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.416143 16182 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.418787 16184 leader_election.cc:290] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 3 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:23.418707 16106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:23.419392 16106 raft_consensus.cc:3049] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:57:23.419710 16106 raft_consensus.cc:738] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:23.419869 16257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429a8b20b2184c82a2222da7025ba714" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:23.420399 16257 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.420495 16106 consensus_queue.cc:260] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.421890 16182 raft_consensus.cc:513] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.422019 16106 raft_consensus.cc:3054] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.424476 16105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6e59a26136604032abe0758f79a8562a"
I20250112 20:57:23.425030 16105 raft_consensus.cc:3049] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:57:23.425308 16105 raft_consensus.cc:738] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6e59a26136604032abe0758f79a8562a, State: Running, Role: LEADER
I20250112 20:57:23.426008 16105 consensus_queue.cc:260] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.426633 16257 raft_consensus.cc:2463] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 3.
I20250112 20:57:23.427126 16105 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.428025 16150 leader_election.cc:304] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f592694e8d94b6b9336474b40fa220c, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:23.429039 16457 raft_consensus.cc:2798] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:57:23.429749 16106 raft_consensus.cc:2463] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 3.
I20250112 20:57:23.430814 16457 raft_consensus.cc:695] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:23.431833 16457 consensus_queue.cc:237] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.433367 16105 raft_consensus.cc:2463] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 3.
I20250112 20:57:23.434367 16182 leader_election.cc:290] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 3 election: Requested vote from peers 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359), 2f592694e8d94b6b9336474b40fa220c (127.15.23.131:34743)
I20250112 20:57:23.434377 16147 leader_election.cc:304] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e59a26136604032abe0758f79a8562a, d5fc77120eda4e978f02d6c6fe3c08a8; no voters:
I20250112 20:57:23.435263 16485 raft_consensus.cc:2798] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:57:23.435178 16258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daed7450d0c64fc784f5b2c8cd139203" candidate_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f592694e8d94b6b9336474b40fa220c"
I20250112 20:57:23.435803 16258 raft_consensus.cc:3054] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:57:23.442462 16258 raft_consensus.cc:2463] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5fc77120eda4e978f02d6c6fe3c08a8 in term 3.
I20250112 20:57:23.448707 16485 raft_consensus.cc:695] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 LEADER]: Becoming Leader. State: Replica: d5fc77120eda4e978f02d6c6fe3c08a8, State: Running, Role: LEADER
I20250112 20:57:23.449468 16485 consensus_queue.cc:237] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } }
I20250112 20:57:23.450215 16007 catalog_manager.cc:5526] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 2 to 3, leader changed from 6e59a26136604032abe0758f79a8562a (127.15.23.129) to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 3 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:23.463096 16007 catalog_manager.cc:5526] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 reported cstate change: term changed from 2 to 3, leader changed from 6e59a26136604032abe0758f79a8562a (127.15.23.129) to d5fc77120eda4e978f02d6c6fe3c08a8 (127.15.23.130). New cstate: current_term: 3 leader_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:23.614298 16182 raft_consensus.cc:1270] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:23.615269 16449 consensus_queue.cc:1035] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:23.629554 16105 raft_consensus.cc:1270] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:23.632546 16492 consensus_queue.cc:1035] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:23.725272 16182 raft_consensus.cc:1270] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:23.726274 16492 consensus_queue.cc:1035] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5fc77120eda4e978f02d6c6fe3c08a8" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:23.737251 16105 raft_consensus.cc:1270] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Refusing update from remote peer 2f592694e8d94b6b9336474b40fa220c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:23.738651 16478 consensus_queue.cc:1035] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:23.841092 16258 raft_consensus.cc:1270] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Refusing update from remote peer d5fc77120eda4e978f02d6c6fe3c08a8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:23.842178 16485 consensus_queue.cc:1035] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:23.851346 16106 raft_consensus.cc:1270] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Refusing update from remote peer d5fc77120eda4e978f02d6c6fe3c08a8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:23.854609 16457 consensus_queue.cc:1035] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:23.911932 16106 raft_consensus.cc:1270] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Refusing update from remote peer d5fc77120eda4e978f02d6c6fe3c08a8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:23.913100 16485 consensus_queue.cc:1035] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e59a26136604032abe0758f79a8562a" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:23.921286 16258 raft_consensus.cc:1270] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Refusing update from remote peer d5fc77120eda4e978f02d6c6fe3c08a8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:57:23.922782 16457 consensus_queue.cc:1035] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f592694e8d94b6b9336474b40fa220c" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:24.121122 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:24.127637 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:24.128003 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:25.128778 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:25.133754 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:25.134146 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:26.134817 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:26.139943 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:26.140290 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:27.141108 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:27.146673 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:27.147212 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:28.147982 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:28.152788 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:28.153226 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:29.153913 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:29.158943 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:29.159399 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:30.160135 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:30.164875 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:30.165201 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:31.165884 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:31.170718 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:31.171080 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:32.171819 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:57:32.176535 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:57:32.176882 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:57:33.205665 15454 auto_leader_rebalancer-test.cc:384] The leader distribution is
I20250112 20:57:33.206241 15454 tablet_server.cc:178] TabletServer@127.15.23.129:0 shutting down...
I20250112 20:57:33.237789 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:57:33.238677 15454 tablet_replica.cc:331] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.239387 15454 raft_consensus.cc:2238] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.239885 15454 raft_consensus.cc:2267] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 6e59a26136604032abe0758f79a8562a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.241678 15454 tablet_replica.cc:331] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.242154 15454 raft_consensus.cc:2238] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.243042 15454 raft_consensus.cc:2267] T 51f70e21e6804833a0e56fd3436032cc P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.244741 15454 tablet_replica.cc:331] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.245460 15454 raft_consensus.cc:2238] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.245946 15454 raft_consensus.cc:2267] T cb464103f740429396ec85635d9853ca P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus is shut down!
W20250112 20:57:33.247056 16221 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
I20250112 20:57:33.248178 15454 tablet_replica.cc:331] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.248785 15454 raft_consensus.cc:2238] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.249255 15454 raft_consensus.cc:2267] T 1e32cc98919f4a5fad3816a08f4a4b18 P 6e59a26136604032abe0758f79a8562a [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:57:33.249902 16221 consensus_peers.cc:487] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c -> Peer 6e59a26136604032abe0758f79a8562a (127.15.23.129:38359): Couldn't send request to peer 6e59a26136604032abe0758f79a8562a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:33.251271 15454 tablet_replica.cc:331] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.251757 15454 raft_consensus.cc:2238] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.252517 15454 raft_consensus.cc:2267] T 45f78ae8287d494781c3d4b9fae23e2f P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.254083 15454 tablet_replica.cc:331] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.254560 15454 raft_consensus.cc:2238] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 4 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.255282 15454 raft_consensus.cc:2267] T 986bdc5230e9412092af88b2f5d1fb5b P 6e59a26136604032abe0758f79a8562a [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.256880 15454 tablet_replica.cc:331] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.257284 15454 raft_consensus.cc:2238] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.257669 15454 raft_consensus.cc:2267] T d969c6c299574510b416acf29125bbfc P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.259258 15454 tablet_replica.cc:331] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.259661 15454 raft_consensus.cc:2238] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.259979 15454 raft_consensus.cc:2267] T 429a8b20b2184c82a2222da7025ba714 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.261449 15454 tablet_replica.cc:331] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.261852 15454 raft_consensus.cc:2238] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.262169 15454 raft_consensus.cc:2267] T daed7450d0c64fc784f5b2c8cd139203 P 6e59a26136604032abe0758f79a8562a [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.263702 15454 tablet_replica.cc:331] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a: stopping tablet replica
I20250112 20:57:33.264056 15454 raft_consensus.cc:2238] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 4 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.264834 15454 raft_consensus.cc:2267] T 85ad275e24e24a3eafa4192cc2e00929 P 6e59a26136604032abe0758f79a8562a [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.307677 15454 tablet_server.cc:195] TabletServer@127.15.23.129:0 shutdown complete.
I20250112 20:57:33.325735 15454 tablet_server.cc:178] TabletServer@127.15.23.130:0 shutting down...
I20250112 20:57:33.350463 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:57:33.351205 15454 tablet_replica.cc:331] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.352138 15454 raft_consensus.cc:2238] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.352890 15454 raft_consensus.cc:2267] T 85ad275e24e24a3eafa4192cc2e00929 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.354761 15454 tablet_replica.cc:331] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.355237 15454 raft_consensus.cc:2238] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.355984 15454 raft_consensus.cc:2267] T daed7450d0c64fc784f5b2c8cd139203 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.357699 15454 tablet_replica.cc:331] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.358156 15454 raft_consensus.cc:2238] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.358947 15454 raft_consensus.cc:2267] T 3ac4a46114fc47ceb7f96990ab3bfa33 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.360530 15454 tablet_replica.cc:331] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.361053 15454 raft_consensus.cc:2238] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.361479 15454 raft_consensus.cc:2267] T 45f78ae8287d494781c3d4b9fae23e2f P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.362920 15454 tablet_replica.cc:331] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.363485 15454 raft_consensus.cc:2238] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.363853 15454 raft_consensus.cc:2267] T 1e32cc98919f4a5fad3816a08f4a4b18 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.365445 15454 tablet_replica.cc:331] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.365846 15454 raft_consensus.cc:2238] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.366233 15454 raft_consensus.cc:2267] T cb464103f740429396ec85635d9853ca P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.367667 15454 tablet_replica.cc:331] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.368078 15454 raft_consensus.cc:2238] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.368510 15454 raft_consensus.cc:2267] T 51f70e21e6804833a0e56fd3436032cc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.370025 15454 tablet_replica.cc:331] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.370429 15454 raft_consensus.cc:2238] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.371208 15454 raft_consensus.cc:2267] T 429a8b20b2184c82a2222da7025ba714 P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.372774 15454 tablet_replica.cc:331] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.373188 15454 raft_consensus.cc:2238] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.373594 15454 raft_consensus.cc:2267] T 986bdc5230e9412092af88b2f5d1fb5b P d5fc77120eda4e978f02d6c6fe3c08a8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.375065 15454 tablet_replica.cc:331] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8: stopping tablet replica
I20250112 20:57:33.375440 15454 raft_consensus.cc:2238] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.375825 15454 raft_consensus.cc:2267] T d969c6c299574510b416acf29125bbfc P d5fc77120eda4e978f02d6c6fe3c08a8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.430076 15454 tablet_server.cc:195] TabletServer@127.15.23.130:0 shutdown complete.
I20250112 20:57:33.447574 15454 tablet_server.cc:178] TabletServer@127.15.23.131:0 shutting down...
I20250112 20:57:33.471307 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:57:33.471899 15454 tablet_replica.cc:331] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.472442 15454 raft_consensus.cc:2238] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.472865 15454 raft_consensus.cc:2267] T daed7450d0c64fc784f5b2c8cd139203 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.474608 15454 tablet_replica.cc:331] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.475065 15454 raft_consensus.cc:2238] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.475525 15454 raft_consensus.cc:2267] T 429a8b20b2184c82a2222da7025ba714 P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.476998 15454 tablet_replica.cc:331] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.477447 15454 raft_consensus.cc:2238] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.477855 15454 raft_consensus.cc:2267] T 3ac4a46114fc47ceb7f96990ab3bfa33 P 2f592694e8d94b6b9336474b40fa220c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.479382 15454 tablet_replica.cc:331] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.479766 15454 raft_consensus.cc:2238] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.480146 15454 raft_consensus.cc:2267] T 45f78ae8287d494781c3d4b9fae23e2f P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.481701 15454 tablet_replica.cc:331] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.482177 15454 raft_consensus.cc:2238] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.482524 15454 raft_consensus.cc:2267] T 85ad275e24e24a3eafa4192cc2e00929 P 2f592694e8d94b6b9336474b40fa220c [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.483876 15454 tablet_replica.cc:331] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.484280 15454 raft_consensus.cc:2238] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.485021 15454 raft_consensus.cc:2267] T 1e32cc98919f4a5fad3816a08f4a4b18 P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.486486 15454 tablet_replica.cc:331] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.486860 15454 raft_consensus.cc:2238] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.487250 15454 raft_consensus.cc:2267] T 51f70e21e6804833a0e56fd3436032cc P 2f592694e8d94b6b9336474b40fa220c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.488763 15454 tablet_replica.cc:331] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.489241 15454 raft_consensus.cc:2238] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.489923 15454 raft_consensus.cc:2267] T cb464103f740429396ec85635d9853ca P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.491345 15454 tablet_replica.cc:331] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.491725 15454 raft_consensus.cc:2238] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:33.492067 15454 raft_consensus.cc:2267] T 986bdc5230e9412092af88b2f5d1fb5b P 2f592694e8d94b6b9336474b40fa220c [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.493480 15454 tablet_replica.cc:331] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c: stopping tablet replica
I20250112 20:57:33.493880 15454 raft_consensus.cc:2238] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.494571 15454 raft_consensus.cc:2267] T d969c6c299574510b416acf29125bbfc P 2f592694e8d94b6b9336474b40fa220c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.538029 15454 tablet_server.cc:195] TabletServer@127.15.23.131:0 shutdown complete.
I20250112 20:57:33.556104 15454 master.cc:537] Master@127.15.23.190:39165 shutting down...
I20250112 20:57:33.577320 15454 raft_consensus.cc:2238] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:57:33.577826 15454 raft_consensus.cc:2267] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:33.578121 15454 tablet_replica.cc:331] T 00000000000000000000000000000000 P f6eff2bdc6564f2898d116fc5d0f4a00: stopping tablet replica
I20250112 20:57:33.615026 15454 master.cc:559] Master@127.15.23.190:39165 shutdown complete.
2f592694e8d94b6b9336474b40fa220c 1
6e59a26136604032abe0758f79a8562a 5
d5fc77120eda4e978f02d6c6fe3c08a8 4
2f592694e8d94b6b9336474b40fa220c 3
6e59a26136604032abe0758f79a8562a 3
d5fc77120eda4e978f02d6c6fe3c08a8 4
2f592694e8d94b6b9336474b40fa220c 1
6e59a26136604032abe0758f79a8562a 8
d5fc77120eda4e978f02d6c6fe3c08a8 1
2f592694e8d94b6b9336474b40fa220c 3
6e59a26136604032abe0758f79a8562a 4
d5fc77120eda4e978f02d6c6fe3c08a8 3
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (30246 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20250112 20:57:33.655927 15454 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.23.190:44397
I20250112 20:57:33.657032 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:57:33.662467 16512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:33.663172 16513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:33.664279 16515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:33.665041 15454 server_base.cc:1034] running on GCE node
I20250112 20:57:33.665900 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:33.666085 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:33.666221 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715453666205 us; error 0 us; skew 500 ppm
I20250112 20:57:33.666682 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:33.673374 15454 webserver.cc:458] Webserver started at http://127.15.23.190:39237/ using document root <none> and password file <none>
I20250112 20:57:33.673808 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:33.673970 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:33.674222 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:33.675366 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/master-0-root/instance:
uuid: "1f263944a9a44fb1abb8091c03bc537a"
format_stamp: "Formatted at 2025-01-12 20:57:33 on dist-test-slave-m2nm"
I20250112 20:57:33.679515 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250112 20:57:33.682410 16520 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:33.683058 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250112 20:57:33.683301 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/master-0-root
uuid: "1f263944a9a44fb1abb8091c03bc537a"
format_stamp: "Formatted at 2025-01-12 20:57:33 on dist-test-slave-m2nm"
I20250112 20:57:33.683540 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:33.699357 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:33.700487 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:33.736111 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.190:44397
I20250112 20:57:33.736199 16571 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.190:44397 every 8 connection(s)
I20250112 20:57:33.740339 16572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:33.749720 16572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a: Bootstrap starting.
I20250112 20:57:33.754096 16572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:33.757797 16572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a: No bootstrap required, opened a new log
I20250112 20:57:33.759631 16572 raft_consensus.cc:357] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f263944a9a44fb1abb8091c03bc537a" member_type: VOTER }
I20250112 20:57:33.760054 16572 raft_consensus.cc:383] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:33.760269 16572 raft_consensus.cc:738] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f263944a9a44fb1abb8091c03bc537a, State: Initialized, Role: FOLLOWER
I20250112 20:57:33.760807 16572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f263944a9a44fb1abb8091c03bc537a" member_type: VOTER }
I20250112 20:57:33.761233 16572 raft_consensus.cc:397] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 20:57:33.761451 16572 raft_consensus.cc:491] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 20:57:33.761703 16572 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:33.765261 16572 raft_consensus.cc:513] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f263944a9a44fb1abb8091c03bc537a" member_type: VOTER }
I20250112 20:57:33.765715 16572 leader_election.cc:304] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f263944a9a44fb1abb8091c03bc537a; no voters:
I20250112 20:57:33.766717 16572 leader_election.cc:290] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 20:57:33.767184 16575 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:33.768503 16575 raft_consensus.cc:695] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 1 LEADER]: Becoming Leader. State: Replica: 1f263944a9a44fb1abb8091c03bc537a, State: Running, Role: LEADER
I20250112 20:57:33.769232 16575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f263944a9a44fb1abb8091c03bc537a" member_type: VOTER }
I20250112 20:57:33.769526 16572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [sys.catalog]: configured and running, proceeding with master startup.
I20250112 20:57:33.773241 16576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f263944a9a44fb1abb8091c03bc537a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f263944a9a44fb1abb8091c03bc537a" member_type: VOTER } }
I20250112 20:57:33.773528 16577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f263944a9a44fb1abb8091c03bc537a. Latest consensus state: current_term: 1 leader_uuid: "1f263944a9a44fb1abb8091c03bc537a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f263944a9a44fb1abb8091c03bc537a" member_type: VOTER } }
I20250112 20:57:33.773962 16576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [sys.catalog]: This master's current role is: LEADER
I20250112 20:57:33.774235 16577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [sys.catalog]: This master's current role is: LEADER
I20250112 20:57:33.777655 16582 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 20:57:33.782156 16582 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 20:57:33.785247 15454 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250112 20:57:33.790617 16582 catalog_manager.cc:1348] Generated new cluster ID: 44ae64917c2e413982ad924dd4d335fb
I20250112 20:57:33.790858 16582 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 20:57:33.815641 16582 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 20:57:33.816787 16582 catalog_manager.cc:1505] Loading token signing keys...
I20250112 20:57:33.833303 16582 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a: Generated new TSK 0
I20250112 20:57:33.833815 16582 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 20:57:33.851879 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:57:33.859283 16593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:33.860560 16594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:33.865381 15454 server_base.cc:1034] running on GCE node
W20250112 20:57:33.866689 16596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:33.867545 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:33.867726 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:33.867875 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715453867858 us; error 0 us; skew 500 ppm
I20250112 20:57:33.868355 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:33.870579 15454 webserver.cc:458] Webserver started at http://127.15.23.129:36631/ using document root <none> and password file <none>
I20250112 20:57:33.871024 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:33.871181 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:33.871425 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:33.872406 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/instance:
uuid: "80770068c3a34f918765f54dbe3a94eb"
format_stamp: "Formatted at 2025-01-12 20:57:33 on dist-test-slave-m2nm"
I20250112 20:57:33.876713 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20250112 20:57:33.879626 16601 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:33.880419 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250112 20:57:33.880685 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
uuid: "80770068c3a34f918765f54dbe3a94eb"
format_stamp: "Formatted at 2025-01-12 20:57:33 on dist-test-slave-m2nm"
I20250112 20:57:33.880940 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:33.901082 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:33.902174 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:33.903472 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:57:33.905586 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:57:33.905771 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:33.905977 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:57:33.906121 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:33.941859 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.129:41463
I20250112 20:57:33.941946 16663 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.129:41463 every 8 connection(s)
I20250112 20:57:33.946095 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:57:33.953657 16668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:33.954934 16669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:33.957062 16671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:33.957863 16664 heartbeater.cc:346] Connected to a master server at 127.15.23.190:44397
I20250112 20:57:33.958194 16664 heartbeater.cc:463] Registering TS with master...
I20250112 20:57:33.958348 15454 server_base.cc:1034] running on GCE node
I20250112 20:57:33.959113 16664 heartbeater.cc:510] Master 127.15.23.190:44397 requested a full tablet report, sending...
I20250112 20:57:33.959376 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:33.959581 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:33.959731 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715453959715 us; error 0 us; skew 500 ppm
I20250112 20:57:33.960311 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:33.960970 16537 ts_manager.cc:194] Registered new tserver with Master: 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:33.962600 15454 webserver.cc:458] Webserver started at http://127.15.23.130:37363/ using document root <none> and password file <none>
I20250112 20:57:33.962765 16537 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50356
I20250112 20:57:33.963070 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:33.963284 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:33.963514 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:33.964512 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/instance:
uuid: "018d03f043b542b9b63491a77523968d"
format_stamp: "Formatted at 2025-01-12 20:57:33 on dist-test-slave-m2nm"
I20250112 20:57:33.968490 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20250112 20:57:33.971400 16676 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:33.972004 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250112 20:57:33.972267 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root
uuid: "018d03f043b542b9b63491a77523968d"
format_stamp: "Formatted at 2025-01-12 20:57:33 on dist-test-slave-m2nm"
I20250112 20:57:33.972528 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:33.986256 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:33.987303 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:33.988533 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:57:33.990469 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:57:33.990646 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:33.990854 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:57:33.990998 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:34.026757 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.130:38087
I20250112 20:57:34.026846 16738 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.130:38087 every 8 connection(s)
I20250112 20:57:34.031018 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:57:34.037954 16742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:34.040717 16743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:34.042176 15454 server_base.cc:1034] running on GCE node
I20250112 20:57:34.042260 16739 heartbeater.cc:346] Connected to a master server at 127.15.23.190:44397
I20250112 20:57:34.042676 16739 heartbeater.cc:463] Registering TS with master...
W20250112 20:57:34.043017 16745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:34.043362 16739 heartbeater.cc:510] Master 127.15.23.190:44397 requested a full tablet report, sending...
I20250112 20:57:34.043877 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:34.044142 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:34.044299 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715454044281 us; error 0 us; skew 500 ppm
I20250112 20:57:34.044988 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:34.045316 16537 ts_manager.cc:194] Registered new tserver with Master: 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:34.046702 16537 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50366
I20250112 20:57:34.047453 15454 webserver.cc:458] Webserver started at http://127.15.23.131:41029/ using document root <none> and password file <none>
I20250112 20:57:34.048084 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:34.048338 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:34.048688 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:34.049845 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/instance:
uuid: "c29b243acf2b4f248606563fc96efc7d"
format_stamp: "Formatted at 2025-01-12 20:57:34 on dist-test-slave-m2nm"
I20250112 20:57:34.053896 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20250112 20:57:34.056806 16750 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:34.057498 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250112 20:57:34.057751 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root
uuid: "c29b243acf2b4f248606563fc96efc7d"
format_stamp: "Formatted at 2025-01-12 20:57:34 on dist-test-slave-m2nm"
I20250112 20:57:34.058027 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:34.077607 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:34.078714 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:34.080065 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:57:34.082413 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:57:34.082626 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:34.082834 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:57:34.082994 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:34.119704 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.131:38713
I20250112 20:57:34.119792 16812 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.131:38713 every 8 connection(s)
I20250112 20:57:34.131883 16813 heartbeater.cc:346] Connected to a master server at 127.15.23.190:44397
I20250112 20:57:34.132210 16813 heartbeater.cc:463] Registering TS with master...
I20250112 20:57:34.132908 16813 heartbeater.cc:510] Master 127.15.23.190:44397 requested a full tablet report, sending...
I20250112 20:57:34.134542 16537 ts_manager.cc:194] Registered new tserver with Master: c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:34.134618 15454 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.012080066s
I20250112 20:57:34.135800 16537 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50368
I20250112 20:57:34.136859 15454 test_util.cc:274] Using random seed: -1816641018
I20250112 20:57:34.158943 16537 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50374:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250112 20:57:34.164986 16537 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250112 20:57:34.363286 16629 tablet_service.cc:1467] Processing CreateTablet for tablet 142fbcc35085432abdb80a4fad031c67 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250112 20:57:34.364744 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142fbcc35085432abdb80a4fad031c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.372551 16695 tablet_service.cc:1467] Processing CreateTablet for tablet b0f35368d307429d8a87ba0c17bb4caf (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250112 20:57:34.374054 16695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0f35368d307429d8a87ba0c17bb4caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.373725 16704 tablet_service.cc:1467] Processing CreateTablet for tablet 142fbcc35085432abdb80a4fad031c67 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250112 20:57:34.375001 16704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142fbcc35085432abdb80a4fad031c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.375123 16778 tablet_service.cc:1467] Processing CreateTablet for tablet 142fbcc35085432abdb80a4fad031c67 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250112 20:57:34.375687 16628 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250112 20:57:34.377732 16627 tablet_service.cc:1467] Processing CreateTablet for tablet 0e510cf639924856a759b20574cb60be (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250112 20:57:34.377785 16777 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250112 20:57:34.379145 16627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e510cf639924856a759b20574cb60be. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.378803 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 0e510cf639924856a759b20574cb60be (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250112 20:57:34.380049 16702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e510cf639924856a759b20574cb60be. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.380000 16776 tablet_service.cc:1467] Processing CreateTablet for tablet 0e510cf639924856a759b20574cb60be (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250112 20:57:34.381705 16626 tablet_service.cc:1467] Processing CreateTablet for tablet d8a2acedfaa2476596c2ec8bb689dcce (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250112 20:57:34.382232 16775 tablet_service.cc:1467] Processing CreateTablet for tablet d8a2acedfaa2476596c2ec8bb689dcce (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250112 20:57:34.383004 16626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a2acedfaa2476596c2ec8bb689dcce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.382828 16700 tablet_service.cc:1467] Processing CreateTablet for tablet 680c875f5d264744aef97e8dd9c304b9 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250112 20:57:34.384074 16700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 680c875f5d264744aef97e8dd9c304b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.384388 16774 tablet_service.cc:1467] Processing CreateTablet for tablet 680c875f5d264744aef97e8dd9c304b9 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250112 20:57:34.385535 16771 tablet_service.cc:1467] Processing CreateTablet for tablet 14c4d77e5fce4c0e92935eb3a168cb77 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250112 20:57:34.380023 16769 tablet_service.cc:1467] Processing CreateTablet for tablet b0f35368d307429d8a87ba0c17bb4caf (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250112 20:57:34.385694 16625 tablet_service.cc:1467] Processing CreateTablet for tablet 680c875f5d264744aef97e8dd9c304b9 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250112 20:57:34.387029 16625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 680c875f5d264744aef97e8dd9c304b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.387537 16772 tablet_service.cc:1467] Processing CreateTablet for tablet abd904e2af24417f8b366444ad932e30 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250112 20:57:34.385694 16699 tablet_service.cc:1467] Processing CreateTablet for tablet 90bf3756af284af8b3ec4d29bbf5b972 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250112 20:57:34.377981 16770 tablet_service.cc:1467] Processing CreateTablet for tablet 1a3f67e9a10d448f93128a7f95344801 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250112 20:57:34.388378 16773 tablet_service.cc:1467] Processing CreateTablet for tablet 90bf3756af284af8b3ec4d29bbf5b972 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250112 20:57:34.389314 16699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90bf3756af284af8b3ec4d29bbf5b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.376439 16778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142fbcc35085432abdb80a4fad031c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.389699 16624 tablet_service.cc:1467] Processing CreateTablet for tablet 90bf3756af284af8b3ec4d29bbf5b972 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250112 20:57:34.390463 16772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd904e2af24417f8b366444ad932e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.390923 16624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90bf3756af284af8b3ec4d29bbf5b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.391047 16774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 680c875f5d264744aef97e8dd9c304b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.391597 16770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3f67e9a10d448f93128a7f95344801. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.393682 16698 tablet_service.cc:1467] Processing CreateTablet for tablet abd904e2af24417f8b366444ad932e30 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250112 20:57:34.394968 16698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd904e2af24417f8b366444ad932e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.396930 16773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90bf3756af284af8b3ec4d29bbf5b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.397781 16623 tablet_service.cc:1467] Processing CreateTablet for tablet abd904e2af24417f8b366444ad932e30 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250112 20:57:34.399003 16623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd904e2af24417f8b366444ad932e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.400924 16771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c4d77e5fce4c0e92935eb3a168cb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.401726 16697 tablet_service.cc:1467] Processing CreateTablet for tablet 14c4d77e5fce4c0e92935eb3a168cb77 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250112 20:57:34.403005 16697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c4d77e5fce4c0e92935eb3a168cb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.404939 16769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0f35368d307429d8a87ba0c17bb4caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.407315 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.407133 16696 tablet_service.cc:1467] Processing CreateTablet for tablet 1a3f67e9a10d448f93128a7f95344801 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250112 20:57:34.408421 16696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3f67e9a10d448f93128a7f95344801. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.410807 16621 tablet_service.cc:1467] Processing CreateTablet for tablet 1a3f67e9a10d448f93128a7f95344801 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250112 20:57:34.411445 16622 tablet_service.cc:1467] Processing CreateTablet for tablet 14c4d77e5fce4c0e92935eb3a168cb77 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250112 20:57:34.412333 16628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.381708 16701 tablet_service.cc:1467] Processing CreateTablet for tablet d8a2acedfaa2476596c2ec8bb689dcce (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250112 20:57:34.423377 16701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a2acedfaa2476596c2ec8bb689dcce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.377751 16703 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250112 20:57:34.425345 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.414975 16620 tablet_service.cc:1467] Processing CreateTablet for tablet b0f35368d307429d8a87ba0c17bb4caf (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250112 20:57:34.417557 16776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e510cf639924856a759b20574cb60be. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.430665 16775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a2acedfaa2476596c2ec8bb689dcce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.430022 16621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3f67e9a10d448f93128a7f95344801. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.432088 16622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c4d77e5fce4c0e92935eb3a168cb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.437820 16620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0f35368d307429d8a87ba0c17bb4caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.466408 16704 tablet_service.cc:1467] Processing CreateTablet for tablet 10e3fd35bff34f4bbaf6763ca8651656 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250112 20:57:34.467728 16704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e3fd35bff34f4bbaf6763ca8651656. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.485354 16778 tablet_service.cc:1467] Processing CreateTablet for tablet 10e3fd35bff34f4bbaf6763ca8651656 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250112 20:57:34.486887 16778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e3fd35bff34f4bbaf6763ca8651656. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.526535 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 43f63edc7cd344fc88b39d2a6b2fbceb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250112 20:57:34.528385 16702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43f63edc7cd344fc88b39d2a6b2fbceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.555632 16829 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:34.562944 16769 tablet_service.cc:1467] Processing CreateTablet for tablet 43f63edc7cd344fc88b39d2a6b2fbceb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250112 20:57:34.564605 16769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43f63edc7cd344fc88b39d2a6b2fbceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.572798 16778 tablet_service.cc:1467] Processing CreateTablet for tablet 153749bb76c146f2ba4e7d07efbc1497 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250112 20:57:34.573812 16774 tablet_service.cc:1467] Processing CreateTablet for tablet dd7c3a6eea9043f4b438c4e28aca966e (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250112 20:57:34.564591 16699 tablet_service.cc:1467] Processing CreateTablet for tablet dd7c3a6eea9043f4b438c4e28aca966e (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250112 20:57:34.577122 16699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd7c3a6eea9043f4b438c4e28aca966e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.583823 16773 tablet_service.cc:1467] Processing CreateTablet for tablet b17fe6b5636e4cc9a2e96751cd7a14c4 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250112 20:57:34.572780 16703 tablet_service.cc:1467] Processing CreateTablet for tablet 153749bb76c146f2ba4e7d07efbc1497 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250112 20:57:34.602782 16776 tablet_service.cc:1467] Processing CreateTablet for tablet 86b544e7e4fa4e419949c833428c406b (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250112 20:57:34.604431 16829 tablet_bootstrap.cc:654] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:34.650671 16828 tablet_bootstrap.cc:492] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:34.659639 16830 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:34.665578 16829 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:34.666752 16829 ts_tablet_manager.cc:1397] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.111s user 0.007s sys 0.013s
I20250112 20:57:34.668831 16830 tablet_bootstrap.cc:654] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:34.669831 16829 raft_consensus.cc:357] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:34.670691 16829 raft_consensus.cc:383] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:34.671066 16829 raft_consensus.cc:738] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:34.671857 16829 consensus_queue.cc:260] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:34.675915 16830 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:34.677315 16620 tablet_service.cc:1467] Processing CreateTablet for tablet 10e3fd35bff34f4bbaf6763ca8651656 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250112 20:57:34.677986 16830 ts_tablet_manager.cc:1397] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.005s
I20250112 20:57:34.681689 16620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e3fd35bff34f4bbaf6763ca8651656. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.682494 16829 ts_tablet_manager.cc:1428] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.015s user 0.006s sys 0.006s
I20250112 20:57:34.684978 16813 heartbeater.cc:502] Master 127.15.23.190:44397 was elected leader, sending a full tablet report...
I20250112 20:57:34.689291 16830 raft_consensus.cc:357] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:34.690835 16830 raft_consensus.cc:383] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:34.691408 16830 raft_consensus.cc:738] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:34.693014 16626 tablet_service.cc:1467] Processing CreateTablet for tablet 43f63edc7cd344fc88b39d2a6b2fbceb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250112 20:57:34.692579 16830 consensus_queue.cc:260] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:34.694387 16773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17fe6b5636e4cc9a2e96751cd7a14c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.696388 16620 tablet_service.cc:1467] Processing CreateTablet for tablet dd7c3a6eea9043f4b438c4e28aca966e (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250112 20:57:34.694383 16626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43f63edc7cd344fc88b39d2a6b2fbceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.706557 16620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd7c3a6eea9043f4b438c4e28aca966e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.715188 16695 tablet_service.cc:1467] Processing CreateTablet for tablet 44e603d3a09141d296df5d386a3b4d11 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250112 20:57:34.717597 16773 tablet_service.cc:1467] Processing CreateTablet for tablet 963bd285f345421581941f10dbe8b90c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250112 20:57:34.718853 16773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 963bd285f345421581941f10dbe8b90c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.719597 16776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b544e7e4fa4e419949c833428c406b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.727214 16664 heartbeater.cc:502] Master 127.15.23.190:44397 was elected leader, sending a full tablet report...
I20250112 20:57:34.727329 16830 ts_tablet_manager.cc:1428] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.048s user 0.014s sys 0.023s
I20250112 20:57:34.727671 16777 tablet_service.cc:1467] Processing CreateTablet for tablet 44e603d3a09141d296df5d386a3b4d11 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250112 20:57:34.728318 16830 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:34.734223 16830 tablet_bootstrap.cc:654] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:34.735810 16778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153749bb76c146f2ba4e7d07efbc1497. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.737638 16626 tablet_service.cc:1467] Processing CreateTablet for tablet 153749bb76c146f2ba4e7d07efbc1497 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250112 20:57:34.738885 16626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153749bb76c146f2ba4e7d07efbc1497. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.742347 16620 tablet_service.cc:1467] Processing CreateTablet for tablet 44e603d3a09141d296df5d386a3b4d11 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250112 20:57:34.743541 16620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e603d3a09141d296df5d386a3b4d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.750782 16698 tablet_service.cc:1467] Processing CreateTablet for tablet b17fe6b5636e4cc9a2e96751cd7a14c4 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250112 20:57:34.753028 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e603d3a09141d296df5d386a3b4d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.757925 16699 tablet_service.cc:1467] Processing CreateTablet for tablet 86b544e7e4fa4e419949c833428c406b (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250112 20:57:34.778174 16829 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:34.760111 16701 tablet_service.cc:1467] Processing CreateTablet for tablet 963bd285f345421581941f10dbe8b90c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250112 20:57:34.763465 16700 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7ad2e8f5064918bd15be2a608c1db4 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
W20250112 20:57:34.786639 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:34.793744 16626 tablet_service.cc:1467] Processing CreateTablet for tablet 86b544e7e4fa4e419949c833428c406b (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250112 20:57:34.795034 16626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b544e7e4fa4e419949c833428c406b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.798202 16829 tablet_bootstrap.cc:654] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:34.790551 16622 tablet_service.cc:1467] Processing CreateTablet for tablet b17fe6b5636e4cc9a2e96751cd7a14c4 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250112 20:57:34.597477 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153749bb76c146f2ba4e7d07efbc1497. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.807128 16701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 963bd285f345421581941f10dbe8b90c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.808457 16774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd7c3a6eea9043f4b438c4e28aca966e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.809289 16699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b544e7e4fa4e419949c833428c406b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.810556 16629 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7ad2e8f5064918bd15be2a608c1db4 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250112 20:57:34.811769 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7ad2e8f5064918bd15be2a608c1db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.816905 16698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17fe6b5636e4cc9a2e96751cd7a14c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.820837 16700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7ad2e8f5064918bd15be2a608c1db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.821808 16695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e603d3a09141d296df5d386a3b4d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.822063 16622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17fe6b5636e4cc9a2e96751cd7a14c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.823058 16830 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:34.823565 16830 ts_tablet_manager.cc:1397] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.095s user 0.013s sys 0.030s
I20250112 20:57:34.827662 16829 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:34.828105 16829 ts_tablet_manager.cc:1397] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.004s
I20250112 20:57:34.828999 16830 raft_consensus.cc:357] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:34.836830 16830 raft_consensus.cc:383] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:34.837102 16830 raft_consensus.cc:738] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:34.837731 16830 consensus_queue.cc:260] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:34.839744 16830 ts_tablet_manager.cc:1428] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.016s user 0.003s sys 0.002s
I20250112 20:57:34.840364 16830 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:34.854323 16830 tablet_bootstrap.cc:654] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:34.816928 16624 tablet_service.cc:1467] Processing CreateTablet for tablet 963bd285f345421581941f10dbe8b90c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250112 20:57:34.869333 16624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 963bd285f345421581941f10dbe8b90c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:34.873888 16665 tablet.cc:2367] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 20:57:34.881048 16829 raft_consensus.cc:357] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:34.881762 16829 raft_consensus.cc:383] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:34.882058 16829 raft_consensus.cc:738] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:34.885047 16829 consensus_queue.cc:260] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:34.897228 16829 ts_tablet_manager.cc:1428] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.069s user 0.004s sys 0.004s
I20250112 20:57:34.898121 16829 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:34.920943 16828 tablet_bootstrap.cc:654] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:34.925769 16774 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7ad2e8f5064918bd15be2a608c1db4 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250112 20:57:34.926867 16774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7ad2e8f5064918bd15be2a608c1db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.927595 16621 tablet_service.cc:1467] Processing CreateTablet for tablet 1b1cfa7b34e542098452925ed942c2bb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250112 20:57:34.928752 16621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1cfa7b34e542098452925ed942c2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.929175 16833 raft_consensus.cc:491] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:34.929610 16833 raft_consensus.cc:513] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:34.930512 16830 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:34.930936 16830 ts_tablet_manager.cc:1397] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.091s user 0.023s sys 0.031s
I20250112 20:57:34.931833 16833 leader_election.cc:290] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:34.933667 16830 raft_consensus.cc:357] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:34.934408 16830 raft_consensus.cc:383] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:34.934691 16830 raft_consensus.cc:738] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:34.935487 16830 consensus_queue.cc:260] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:34.937129 16700 tablet_service.cc:1467] Processing CreateTablet for tablet 1b1cfa7b34e542098452925ed942c2bb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250112 20:57:34.937868 16830 ts_tablet_manager.cc:1428] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.007s user 0.001s sys 0.006s
I20250112 20:57:34.938495 16700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1cfa7b34e542098452925ed942c2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.948698 16622 tablet_service.cc:1467] Processing CreateTablet for tablet f3b1a80e42504299a3970ea4437b8a13 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250112 20:57:34.949811 16699 tablet_service.cc:1467] Processing CreateTablet for tablet f3b1a80e42504299a3970ea4437b8a13 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250112 20:57:34.950332 16629 tablet_service.cc:1467] Processing CreateTablet for tablet d979f5162b594f329ba9439c121741de (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250112 20:57:34.931933 16829 tablet_bootstrap.cc:654] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:34.951182 16699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b1a80e42504299a3970ea4437b8a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.965308 16830 tablet_bootstrap.cc:492] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:34.969638 16777 tablet_service.cc:1467] Processing CreateTablet for tablet 1b1cfa7b34e542098452925ed942c2bb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250112 20:57:34.970221 16771 tablet_service.cc:1467] Processing CreateTablet for tablet f3b1a80e42504299a3970ea4437b8a13 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250112 20:57:34.970809 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1cfa7b34e542098452925ed942c2bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.971621 16771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b1a80e42504299a3970ea4437b8a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.972385 16776 tablet_service.cc:1467] Processing CreateTablet for tablet d979f5162b594f329ba9439c121741de (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250112 20:57:34.973506 16776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d979f5162b594f329ba9439c121741de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.976823 16830 tablet_bootstrap.cc:654] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:34.977048 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d979f5162b594f329ba9439c121741de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.981350 16625 tablet_service.cc:1467] Processing CreateTablet for tablet 132239c0c07a4c2f9334e84188f329b3 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250112 20:57:34.989403 16625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 132239c0c07a4c2f9334e84188f329b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:34.993108 16698 tablet_service.cc:1467] Processing CreateTablet for tablet d979f5162b594f329ba9439c121741de (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250112 20:57:34.994267 16698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d979f5162b594f329ba9439c121741de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.002735 16701 tablet_service.cc:1467] Processing CreateTablet for tablet 132239c0c07a4c2f9334e84188f329b3 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250112 20:57:35.003935 16701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 132239c0c07a4c2f9334e84188f329b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.005913 16627 tablet_service.cc:1467] Processing CreateTablet for tablet e19f4890a44f4f088549ad693bafbf8d (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250112 20:57:35.007949 16628 tablet_service.cc:1467] Processing CreateTablet for tablet 3c5b79af971640d891e05584ff40dd7b (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250112 20:57:35.010483 16704 tablet_service.cc:1467] Processing CreateTablet for tablet e19f4890a44f4f088549ad693bafbf8d (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250112 20:57:35.011679 16704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e19f4890a44f4f088549ad693bafbf8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.017439 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 3c5b79af971640d891e05584ff40dd7b (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250112 20:57:35.021263 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:35.022097 16627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e19f4890a44f4f088549ad693bafbf8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.022753 16828 tablet_bootstrap.cc:492] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:35.023211 16828 ts_tablet_manager.cc:1397] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.373s user 0.015s sys 0.021s
I20250112 20:57:35.023296 16702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5b79af971640d891e05584ff40dd7b. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.022753 16751 leader_election.cc:343] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.024983 16622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b1a80e42504299a3970ea4437b8a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.026144 16628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5b79af971640d891e05584ff40dd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.029242 16773 tablet_service.cc:1467] Processing CreateTablet for tablet 3c5b79af971640d891e05584ff40dd7b (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250112 20:57:35.030402 16773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c5b79af971640d891e05584ff40dd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.033536 16774 tablet_service.cc:1467] Processing CreateTablet for tablet e19f4890a44f4f088549ad693bafbf8d (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250112 20:57:35.034619 16774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e19f4890a44f4f088549ad693bafbf8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.034824 16772 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250112 20:57:35.035254 16697 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250112 20:57:35.036017 16772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.036444 16697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.037446 16778 tablet_service.cc:1467] Processing CreateTablet for tablet a5ffa8e639d640a3b7f19d9857148f40 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250112 20:57:35.049335 16778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ffa8e639d640a3b7f19d9857148f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.049417 16739 heartbeater.cc:502] Master 127.15.23.190:44397 was elected leader, sending a full tablet report...
I20250112 20:57:35.038755 16703 tablet_service.cc:1467] Processing CreateTablet for tablet a5ffa8e639d640a3b7f19d9857148f40 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250112 20:57:35.053339 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ffa8e639d640a3b7f19d9857148f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.040036 16696 tablet_service.cc:1467] Processing CreateTablet for tablet 7d37ea77e5934355b580ed367659c16c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250112 20:57:35.040652 16769 tablet_service.cc:1467] Processing CreateTablet for tablet 132239c0c07a4c2f9334e84188f329b3 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250112 20:57:35.041059 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:35.062054 16621 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250112 20:57:35.063396 16621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.064885 16769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 132239c0c07a4c2f9334e84188f329b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.067751 16629 tablet_service.cc:1467] Processing CreateTablet for tablet 7d37ea77e5934355b580ed367659c16c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250112 20:57:35.069000 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d37ea77e5934355b580ed367659c16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.069551 16828 raft_consensus.cc:357] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.070259 16828 raft_consensus.cc:383] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.070519 16828 raft_consensus.cc:738] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.071230 16828 consensus_queue.cc:260] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.074889 16696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d37ea77e5934355b580ed367659c16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.076176 16625 tablet_service.cc:1467] Processing CreateTablet for tablet a5ffa8e639d640a3b7f19d9857148f40 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250112 20:57:35.077396 16625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ffa8e639d640a3b7f19d9857148f40. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.082357 16752 leader_election.cc:343] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.084317 16752 leader_election.cc:304] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:35.086299 16701 tablet_service.cc:1467] Processing CreateTablet for tablet 6c34b0c62d424c9e9f764d74f6baf1ba (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250112 20:57:35.086998 16833 raft_consensus.cc:2743] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.087697 16701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c34b0c62d424c9e9f764d74f6baf1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.103191 16829 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:35.103641 16829 ts_tablet_manager.cc:1397] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.206s user 0.016s sys 0.000s
I20250112 20:57:35.106235 16829 raft_consensus.cc:357] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.106911 16829 raft_consensus.cc:383] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.107182 16829 raft_consensus.cc:738] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.109457 16834 raft_consensus.cc:491] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.109879 16834 raft_consensus.cc:513] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.110536 16772 tablet_service.cc:1467] Processing CreateTablet for tablet 7d37ea77e5934355b580ed367659c16c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250112 20:57:35.111672 16772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d37ea77e5934355b580ed367659c16c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.112192 16834 leader_election.cc:290] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:35.108681 16776 tablet_service.cc:1467] Processing CreateTablet for tablet 6c34b0c62d424c9e9f764d74f6baf1ba (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250112 20:57:35.118983 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 7a13e4117fd64b15a4cd60943e638999 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250112 20:57:35.125968 16834 raft_consensus.cc:491] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.126333 16702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a13e4117fd64b15a4cd60943e638999. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.126415 16834 raft_consensus.cc:513] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.130620 16830 tablet_bootstrap.cc:492] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:35.131047 16830 ts_tablet_manager.cc:1397] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.166s user 0.000s sys 0.026s
I20250112 20:57:35.131685 16770 tablet_service.cc:1467] Processing CreateTablet for tablet 7a13e4117fd64b15a4cd60943e638999 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250112 20:57:35.126585 16777 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3206c0399f4057928460c7a6f04912 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250112 20:57:35.132781 16845 raft_consensus.cc:491] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.123339 16776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c34b0c62d424c9e9f764d74f6baf1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.133237 16845 raft_consensus.cc:513] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.128379 16834 leader_election.cc:290] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:35.118983 16828 ts_tablet_manager.cc:1428] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.095s user 0.006s sys 0.003s
I20250112 20:57:35.137807 16845 leader_election.cc:290] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:35.150671 16770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a13e4117fd64b15a4cd60943e638999. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.157001 16828 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:35.122548 16829 consensus_queue.cc:260] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.165549 16624 tablet_service.cc:1467] Processing CreateTablet for tablet 6c34b0c62d424c9e9f764d74f6baf1ba (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250112 20:57:35.168727 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3206c0399f4057928460c7a6f04912. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.169387 16624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c34b0c62d424c9e9f764d74f6baf1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.173100 16833 raft_consensus.cc:491] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.173538 16833 raft_consensus.cc:513] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.175048 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:35.175401 16833 leader_election.cc:290] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:35.176806 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a2acedfaa2476596c2ec8bb689dcce" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:35.178117 16602 leader_election.cc:343] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:35.178740 16602 leader_election.cc:343] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.178376 16698 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3206c0399f4057928460c7a6f04912 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250112 20:57:35.179749 16698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3206c0399f4057928460c7a6f04912. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.192070 16629 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3206c0399f4057928460c7a6f04912 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250112 20:57:35.193567 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3206c0399f4057928460c7a6f04912. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.201156 16830 raft_consensus.cc:357] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.201892 16830 raft_consensus.cc:383] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.202164 16830 raft_consensus.cc:738] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.202797 16830 consensus_queue.cc:260] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.223752 16828 tablet_bootstrap.cc:654] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:35.235975 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:35.245949 16829 ts_tablet_manager.cc:1428] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.142s user 0.011s sys 0.013s
I20250112 20:57:35.246928 16829 tablet_bootstrap.cc:492] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:35.256625 16703 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9eab84e9c44a4aae8715e7735721d8 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250112 20:57:35.258952 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b9eab84e9c44a4aae8715e7735721d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.261469 16853 raft_consensus.cc:491] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.261781 16830 ts_tablet_manager.cc:1428] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.130s user 0.005s sys 0.004s
I20250112 20:57:35.262001 16853 raft_consensus.cc:513] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.262748 16830 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:35.266863 16828 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:35.267730 16828 ts_tablet_manager.cc:1397] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.111s user 0.011s sys 0.007s
I20250112 20:57:35.267767 16773 tablet_service.cc:1467] Processing CreateTablet for tablet 5ffbc5c73802406893d78f89cf5c145c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250112 20:57:35.267189 16853 leader_election.cc:290] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:35.269083 16773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffbc5c73802406893d78f89cf5c145c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.271991 16834 raft_consensus.cc:491] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.272421 16834 raft_consensus.cc:513] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.273236 16697 tablet_service.cc:1467] Processing CreateTablet for tablet df9c04a57cac449e8efae9c7689ad690 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250112 20:57:35.274385 16697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9c04a57cac449e8efae9c7689ad690. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.275178 16829 tablet_bootstrap.cc:654] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:35.275738 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:35.277058 16602 leader_election.cc:343] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.274300 16834 leader_election.cc:290] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:35.281486 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:35.282851 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:35.283998 16751 leader_election.cc:343] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:35.284910 16751 leader_election.cc:343] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.285583 16624 tablet_service.cc:1467] Processing CreateTablet for tablet df9c04a57cac449e8efae9c7689ad690 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250112 20:57:35.286254 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bf3756af284af8b3ec4d29bbf5b972" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:35.286875 16624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9c04a57cac449e8efae9c7689ad690. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.287628 16678 leader_election.cc:343] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.285583 16629 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9eab84e9c44a4aae8715e7735721d8 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250112 20:57:35.294888 16700 tablet_service.cc:1467] Processing CreateTablet for tablet c1a1a7f345014fa7aff0f0cc0160789f (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250112 20:57:35.295640 16704 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10f3f7e0f74c9f988fa01e7843e961 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250112 20:57:35.296406 16700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a1a7f345014fa7aff0f0cc0160789f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.296098 16701 tablet_service.cc:1467] Processing CreateTablet for tablet 81567654a26f435795e38a761d26bfa7 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250112 20:57:35.296944 16704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c10f3f7e0f74c9f988fa01e7843e961. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.294107 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 5ffbc5c73802406893d78f89cf5c145c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250112 20:57:35.297114 16770 tablet_service.cc:1467] Processing CreateTablet for tablet df9c04a57cac449e8efae9c7689ad690 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250112 20:57:35.298195 16702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffbc5c73802406893d78f89cf5c145c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.300911 16701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81567654a26f435795e38a761d26bfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.297803 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b9eab84e9c44a4aae8715e7735721d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.313158 16770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9c04a57cac449e8efae9c7689ad690. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.314757 16778 tablet_service.cc:1467] Processing CreateTablet for tablet b1d0b3432b644821ba556f59a893d40a (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250112 20:57:35.315343 16771 tablet_service.cc:1467] Processing CreateTablet for tablet 0b0ecdb5a6684883976581f48f328035 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250112 20:57:35.315668 16772 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10f3f7e0f74c9f988fa01e7843e961 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250112 20:57:35.316128 16769 tablet_service.cc:1467] Processing CreateTablet for tablet 305481b596b746b1a03df8d76e4057fb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250112 20:57:35.321348 16778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d0b3432b644821ba556f59a893d40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.322490 16771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b0ecdb5a6684883976581f48f328035. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.313807 16774 tablet_service.cc:1467] Processing CreateTablet for tablet 81567654a26f435795e38a761d26bfa7 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250112 20:57:35.317051 16775 tablet_service.cc:1467] Processing CreateTablet for tablet 8e161101273b4b61a7235af454d77ca1 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250112 20:57:35.317946 16776 tablet_service.cc:1467] Processing CreateTablet for tablet c1a1a7f345014fa7aff0f0cc0160789f (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250112 20:57:35.319324 16622 tablet_service.cc:1467] Processing CreateTablet for tablet 5ffbc5c73802406893d78f89cf5c145c (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250112 20:57:35.326762 16769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305481b596b746b1a03df8d76e4057fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.327091 16622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffbc5c73802406893d78f89cf5c145c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.331341 16830 tablet_bootstrap.cc:654] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:35.325138 16777 tablet_service.cc:1467] Processing CreateTablet for tablet 1b9eab84e9c44a4aae8715e7735721d8 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250112 20:57:35.335022 16703 tablet_service.cc:1467] Processing CreateTablet for tablet b1d0b3432b644821ba556f59a893d40a (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250112 20:57:35.335150 16625 tablet_service.cc:1467] Processing CreateTablet for tablet 81567654a26f435795e38a761d26bfa7 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250112 20:57:35.336153 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d0b3432b644821ba556f59a893d40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.336274 16625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81567654a26f435795e38a761d26bfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.341058 16776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a1a7f345014fa7aff0f0cc0160789f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.341711 16696 tablet_service.cc:1467] Processing CreateTablet for tablet 305481b596b746b1a03df8d76e4057fb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250112 20:57:35.342849 16696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305481b596b746b1a03df8d76e4057fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.345064 16775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e161101273b4b61a7235af454d77ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.284226 16828 raft_consensus.cc:357] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:35.346038 16828 raft_consensus.cc:383] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.346338 16828 raft_consensus.cc:738] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.346771 16773 tablet_service.cc:1467] Processing CreateTablet for tablet 1696e724d24e4ebbb185476ec5ff8c09 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250112 20:57:35.346982 16828 consensus_queue.cc:260] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:35.347772 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:35.350488 16639 raft_consensus.cc:2463] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:35.350425 16620 tablet_service.cc:1467] Processing CreateTablet for tablet 305481b596b746b1a03df8d76e4057fb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250112 20:57:35.348910 16627 tablet_service.cc:1467] Processing CreateTablet for tablet c1a1a7f345014fa7aff0f0cc0160789f (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250112 20:57:35.351538 16620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305481b596b746b1a03df8d76e4057fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.352085 16627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a1a7f345014fa7aff0f0cc0160789f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.320475 16621 tablet_service.cc:1467] Processing CreateTablet for tablet 7a13e4117fd64b15a4cd60943e638999 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250112 20:57:35.349166 16628 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10f3f7e0f74c9f988fa01e7843e961 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250112 20:57:35.353374 16621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a13e4117fd64b15a4cd60943e638999. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.353226 16623 tablet_service.cc:1467] Processing CreateTablet for tablet 0b0ecdb5a6684883976581f48f328035 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250112 20:57:35.354067 16628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c10f3f7e0f74c9f988fa01e7843e961. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.354630 16623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b0ecdb5a6684883976581f48f328035. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.364930 16752 leader_election.cc:343] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.365415 16752 leader_election.cc:304] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:35.366165 16834 raft_consensus.cc:491] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.366371 16853 raft_consensus.cc:2743] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.345870 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b9eab84e9c44a4aae8715e7735721d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.349732 16626 tablet_service.cc:1467] Processing CreateTablet for tablet b1d0b3432b644821ba556f59a893d40a (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250112 20:57:35.366554 16752 leader_election.cc:304] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:35.366611 16834 raft_consensus.cc:513] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.369689 16834 leader_election.cc:290] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:35.371204 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:35.372494 16602 leader_election.cc:343] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.373318 16626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d0b3432b644821ba556f59a893d40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.376791 16853 raft_consensus.cc:2798] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:35.377133 16853 raft_consensus.cc:491] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:35.377480 16853 raft_consensus.cc:3054] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:35.383759 16853 raft_consensus.cc:513] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.385227 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:35.385985 16788 raft_consensus.cc:2388] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80770068c3a34f918765f54dbe3a94eb in current term 1: Already voted for candidate c29b243acf2b4f248606563fc96efc7d in this term.
I20250112 20:57:35.386955 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a2acedfaa2476596c2ec8bb689dcce" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:35.388128 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:35.389276 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:35.390650 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
W20250112 20:57:35.391861 16751 leader_election.cc:343] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.392395 16853 leader_election.cc:290] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:35.393725 16605 leader_election.cc:304] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:35.393798 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:35.394459 16639 raft_consensus.cc:3054] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:35.401398 16639 raft_consensus.cc:2463] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:35.402122 16828 ts_tablet_manager.cc:1428] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.134s user 0.004s sys 0.002s
I20250112 20:57:35.402544 16834 raft_consensus.cc:2743] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.403093 16828 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
W20250112 20:57:35.403656 16605 leader_election.cc:343] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.404088 16605 leader_election.cc:304] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:35.404788 16834 raft_consensus.cc:2743] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.405596 16772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c10f3f7e0f74c9f988fa01e7843e961. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.405417 16620 tablet_service.cc:1467] Processing CreateTablet for tablet e23f9778f8494e11b9d942035720e363 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250112 20:57:35.406642 16620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e23f9778f8494e11b9d942035720e363. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.408852 16774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81567654a26f435795e38a761d26bfa7. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.409150 16605 leader_election.cc:343] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.409561 16605 leader_election.cc:304] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:35.410264 16834 raft_consensus.cc:2743] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.411062 16773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1696e724d24e4ebbb185476ec5ff8c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.413070 16629 tablet_service.cc:1467] Processing CreateTablet for tablet 8e161101273b4b61a7235af454d77ca1 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250112 20:57:35.415599 16698 tablet_service.cc:1467] Processing CreateTablet for tablet 1696e724d24e4ebbb185476ec5ff8c09 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250112 20:57:35.416204 16695 tablet_service.cc:1467] Processing CreateTablet for tablet aab6f0844afa43d8b7af8e3705d09c40 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250112 20:57:35.417060 16700 tablet_service.cc:1467] Processing CreateTablet for tablet 807445e11d91471bbfa7a55fcb99f774 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250112 20:57:35.418054 16699 tablet_service.cc:1467] Processing CreateTablet for tablet 03cbd278697d4f6780211b21028f6d18 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250112 20:57:35.426016 16699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03cbd278697d4f6780211b21028f6d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.422226 16769 tablet_service.cc:1467] Processing CreateTablet for tablet aab6f0844afa43d8b7af8e3705d09c40 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250112 20:57:35.429917 16769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab6f0844afa43d8b7af8e3705d09c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.431561 16704 tablet_service.cc:1467] Processing CreateTablet for tablet 9a21c93f28fc4ab3be409b95204b86dc (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250112 20:57:35.433000 16778 tablet_service.cc:1467] Processing CreateTablet for tablet 807445e11d91471bbfa7a55fcb99f774 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250112 20:57:35.434166 16778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 807445e11d91471bbfa7a55fcb99f774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.434397 16628 tablet_service.cc:1467] Processing CreateTablet for tablet 807445e11d91471bbfa7a55fcb99f774 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250112 20:57:35.435529 16628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 807445e11d91471bbfa7a55fcb99f774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.439493 16624 tablet_service.cc:1467] Processing CreateTablet for tablet 1696e724d24e4ebbb185476ec5ff8c09 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250112 20:57:35.443814 16703 tablet_service.cc:1467] Processing CreateTablet for tablet e23f9778f8494e11b9d942035720e363 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250112 20:57:35.436867 16752 leader_election.cc:304] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:35.436992 16700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 807445e11d91471bbfa7a55fcb99f774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.445324 16853 raft_consensus.cc:2798] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:35.445434 16695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab6f0844afa43d8b7af8e3705d09c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.446066 16698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1696e724d24e4ebbb185476ec5ff8c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.448870 16704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a21c93f28fc4ab3be409b95204b86dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.453460 16853 raft_consensus.cc:695] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:35.454325 16853 consensus_queue.cc:237] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:35.455889 16605 leader_election.cc:343] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.456326 16605 leader_election.cc:304] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:35.432124 16771 tablet_service.cc:1467] Processing CreateTablet for tablet 03cbd278697d4f6780211b21028f6d18 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250112 20:57:35.460861 16834 raft_consensus.cc:2743] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.469338 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 8e161101273b4b61a7235af454d77ca1 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250112 20:57:35.457633 16771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03cbd278697d4f6780211b21028f6d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.482764 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e23f9778f8494e11b9d942035720e363. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.483256 16697 tablet_service.cc:1467] Processing CreateTablet for tablet 0b0ecdb5a6684883976581f48f328035 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250112 20:57:35.484776 16830 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:35.485224 16830 ts_tablet_manager.cc:1397] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.223s user 0.010s sys 0.014s
I20250112 20:57:35.488786 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e161101273b4b61a7235af454d77ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.487990 16830 raft_consensus.cc:357] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.490841 16830 raft_consensus.cc:383] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.491149 16830 raft_consensus.cc:738] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.492892 16624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1696e724d24e4ebbb185476ec5ff8c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.495020 16697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b0ecdb5a6684883976581f48f328035. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.498581 16702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e161101273b4b61a7235af454d77ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.498144 16830 consensus_queue.cc:260] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.500752 16627 tablet_service.cc:1467] Processing CreateTablet for tablet aab6f0844afa43d8b7af8e3705d09c40 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250112 20:57:35.501945 16627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab6f0844afa43d8b7af8e3705d09c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.503140 16828 tablet_bootstrap.cc:654] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:35.500774 16830 ts_tablet_manager.cc:1428] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.015s user 0.008s sys 0.001s
I20250112 20:57:35.513592 16830 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:35.516203 16625 tablet_service.cc:1467] Processing CreateTablet for tablet 03cbd278697d4f6780211b21028f6d18 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250112 20:57:35.517704 16625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03cbd278697d4f6780211b21028f6d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.522181 16622 tablet_service.cc:1467] Processing CreateTablet for tablet 9a21c93f28fc4ab3be409b95204b86dc (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250112 20:57:35.523669 16622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a21c93f28fc4ab3be409b95204b86dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.510046 16703 tablet_service.cc:1467] Processing CreateTablet for tablet f021c2fa4d3143ff899283a33cb24538 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250112 20:57:35.539819 16829 tablet_bootstrap.cc:492] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:35.539543 16773 tablet_service.cc:1467] Processing CreateTablet for tablet e23f9778f8494e11b9d942035720e363 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250112 20:57:35.540274 16829 ts_tablet_manager.cc:1397] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.294s user 0.005s sys 0.012s
I20250112 20:57:35.542939 16829 raft_consensus.cc:357] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.543622 16829 raft_consensus.cc:383] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.543890 16829 raft_consensus.cc:738] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.543900 16620 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2477026d8d4fac945714c894cca485 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250112 20:57:35.557881 16620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d2477026d8d4fac945714c894cca485. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.562283 16621 tablet_service.cc:1467] Processing CreateTablet for tablet cc2820c3d5974dfcb2230e18411e6493 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250112 20:57:35.563500 16621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc2820c3d5974dfcb2230e18411e6493. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.570171 16828 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:35.570735 16828 ts_tablet_manager.cc:1397] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.168s user 0.013s sys 0.007s
I20250112 20:57:35.573490 16828 raft_consensus.cc:357] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.574280 16828 raft_consensus.cc:383] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.574630 16828 raft_consensus.cc:738] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.576930 16623 tablet_service.cc:1467] Processing CreateTablet for tablet f021c2fa4d3143ff899283a33cb24538 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250112 20:57:35.578053 16777 tablet_service.cc:1467] Processing CreateTablet for tablet 9a21c93f28fc4ab3be409b95204b86dc (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250112 20:57:35.588764 16701 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2477026d8d4fac945714c894cca485 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250112 20:57:35.591470 16770 tablet_service.cc:1467] Processing CreateTablet for tablet f021c2fa4d3143ff899283a33cb24538 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250112 20:57:35.544536 16829 consensus_queue.cc:260] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.591229 16697 tablet_service.cc:1467] Processing CreateTablet for tablet cc2820c3d5974dfcb2230e18411e6493 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250112 20:57:35.593384 16830 tablet_bootstrap.cc:654] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:35.596482 16624 tablet_service.cc:1467] Processing CreateTablet for tablet 04890c40fe454c4a91d3d12e4104200a (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250112 20:57:35.599864 16773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e23f9778f8494e11b9d942035720e363. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.614440 16626 tablet_service.cc:1467] Processing CreateTablet for tablet e1773dc29c9e4d83a01c6b56f7b277d5 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250112 20:57:35.620029 16622 tablet_service.cc:1467] Processing CreateTablet for tablet abeaacbf8c794ec89adddeb5f906f096 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250112 20:57:35.627898 16829 ts_tablet_manager.cc:1428] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.087s user 0.008s sys 0.000s
I20250112 20:57:35.627168 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f021c2fa4d3143ff899283a33cb24538. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.633374 16697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc2820c3d5974dfcb2230e18411e6493. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.636853 16701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d2477026d8d4fac945714c894cca485. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.637321 16699 tablet_service.cc:1467] Processing CreateTablet for tablet e1773dc29c9e4d83a01c6b56f7b277d5 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250112 20:57:35.642800 16626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1773dc29c9e4d83a01c6b56f7b277d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.643728 16623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f021c2fa4d3143ff899283a33cb24538. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.645767 16704 tablet_service.cc:1467] Processing CreateTablet for tablet abeaacbf8c794ec89adddeb5f906f096 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250112 20:57:35.647522 16769 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2477026d8d4fac945714c894cca485 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250112 20:57:35.639653 16699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1773dc29c9e4d83a01c6b56f7b277d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.658219 16698 tablet_service.cc:1467] Processing CreateTablet for tablet 04890c40fe454c4a91d3d12e4104200a (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250112 20:57:35.660540 16621 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdb0f016b8946c582e26ab6e02cb9b1 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250112 20:57:35.661686 16621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bdb0f016b8946c582e26ab6e02cb9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.662236 16622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abeaacbf8c794ec89adddeb5f906f096. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.663120 16536 catalog_manager.cc:5526] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:35.666566 16698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04890c40fe454c4a91d3d12e4104200a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.667450 16624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04890c40fe454c4a91d3d12e4104200a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.671178 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bf3756af284af8b3ec4d29bbf5b972" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:35.672461 16680 leader_election.cc:343] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.676400 16680 leader_election.cc:304] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:35.677994 16830 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:35.678105 16829 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:35.678457 16830 ts_tablet_manager.cc:1397] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.165s user 0.023s sys 0.014s
I20250112 20:57:35.678612 16772 tablet_service.cc:1467] Processing CreateTablet for tablet e1773dc29c9e4d83a01c6b56f7b277d5 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250112 20:57:35.679802 16834 raft_consensus.cc:491] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.674355 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a21c93f28fc4ab3be409b95204b86dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.680172 16700 tablet_service.cc:1467] Processing CreateTablet for tablet 4799def22f754c9881fd8245eb8f75b3 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250112 20:57:35.680269 16834 raft_consensus.cc:513] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.681144 16770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f021c2fa4d3143ff899283a33cb24538. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.682499 16834 leader_election.cc:290] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:35.683929 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:35.685711 16602 leader_election.cc:343] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.685845 16775 tablet_service.cc:1467] Processing CreateTablet for tablet abeaacbf8c794ec89adddeb5f906f096 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250112 20:57:35.686472 16772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1773dc29c9e4d83a01c6b56f7b277d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.686520 16625 tablet_service.cc:1467] Processing CreateTablet for tablet 4799def22f754c9881fd8245eb8f75b3 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250112 20:57:35.687815 16625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4799def22f754c9881fd8245eb8f75b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.688163 16775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abeaacbf8c794ec89adddeb5f906f096. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.616433 16828 consensus_queue.cc:260] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.689055 16845 raft_consensus.cc:491] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.689492 16845 raft_consensus.cc:513] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.690158 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:35.692175 16605 leader_election.cc:343] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.699074 16830 raft_consensus.cc:357] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.693181 16769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d2477026d8d4fac945714c894cca485. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.699841 16830 raft_consensus.cc:383] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.700181 16830 raft_consensus.cc:738] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.699374 16605 leader_election.cc:304] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:35.700409 16695 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdb0f016b8946c582e26ab6e02cb9b1 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250112 20:57:35.701151 16834 raft_consensus.cc:2743] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.700899 16830 consensus_queue.cc:260] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.676484 16704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abeaacbf8c794ec89adddeb5f906f096. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.708774 16773 tablet_service.cc:1467] Processing CreateTablet for tablet 4799def22f754c9881fd8245eb8f75b3 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250112 20:57:35.710002 16773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4799def22f754c9881fd8245eb8f75b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.709790 16628 tablet_service.cc:1467] Processing CreateTablet for tablet 82159eb8aadb4973b65d7edc4f417d84 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250112 20:57:35.710937 16628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82159eb8aadb4973b65d7edc4f417d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.698474 16774 tablet_service.cc:1467] Processing CreateTablet for tablet 2bdb0f016b8946c582e26ab6e02cb9b1 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250112 20:57:35.713240 16695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bdb0f016b8946c582e26ab6e02cb9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.713737 16774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bdb0f016b8946c582e26ab6e02cb9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.686578 16778 tablet_service.cc:1467] Processing CreateTablet for tablet 04890c40fe454c4a91d3d12e4104200a (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250112 20:57:35.694252 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e510cf639924856a759b20574cb60be" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:35.695029 16845 leader_election.cc:290] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:35.697935 16620 tablet_service.cc:1467] Processing CreateTablet for tablet 1f61fd1d8bf44f64b73ade11c0224df0 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250112 20:57:35.698478 16623 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef14661bd7346f68101fe2a0fb064bb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250112 20:57:35.671955 16776 tablet_service.cc:1467] Processing CreateTablet for tablet cc2820c3d5974dfcb2230e18411e6493 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250112 20:57:35.698944 16829 tablet_bootstrap.cc:654] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:35.720837 16678 leader_election.cc:343] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.735072 16778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04890c40fe454c4a91d3d12e4104200a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.725209 16845 raft_consensus.cc:2743] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.762881 16777 tablet_service.cc:1467] Processing CreateTablet for tablet 1f61fd1d8bf44f64b73ade11c0224df0 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250112 20:57:35.769845 16830 ts_tablet_manager.cc:1428] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.091s user 0.010s sys 0.008s
I20250112 20:57:35.734771 16629 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0f7e20973149ed977de7cdd902fc39 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
W20250112 20:57:35.779975 16680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.131:38713, user_credentials={real_user=slave}} blocked reactor thread for 107615us
I20250112 20:57:35.786276 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e510cf639924856a759b20574cb60be" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:35.720857 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef14661bd7346f68101fe2a0fb064bb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250112 20:57:35.789263 16776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc2820c3d5974dfcb2230e18411e6493. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.792210 16809 debug-util.cc:398] Leaking SignalData structure 0x7b08000a7a40 after lost signal to thread 15457
I20250112 20:57:35.792795 16620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f61fd1d8bf44f64b73ade11c0224df0. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.793145 16809 debug-util.cc:398] Leaking SignalData structure 0x7b08000a0b40 after lost signal to thread 16571
I20250112 20:57:35.796990 16696 tablet_service.cc:1467] Processing CreateTablet for tablet 1f61fd1d8bf44f64b73ade11c0224df0 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250112 20:57:35.798005 16629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0f7e20973149ed977de7cdd902fc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.800705 16700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4799def22f754c9881fd8245eb8f75b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.804950 16623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ef14661bd7346f68101fe2a0fb064bb. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.806681 16680 leader_election.cc:343] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.807345 16680 leader_election.cc:304] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:35.808389 16845 raft_consensus.cc:491] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:35.808843 16702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ef14661bd7346f68101fe2a0fb064bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.808880 16845 raft_consensus.cc:513] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:35.810981 16845 leader_election.cc:290] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713), 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:35.811468 16845 raft_consensus.cc:2743] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:35.812162 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:35.812222 16703 tablet_service.cc:1467] Processing CreateTablet for tablet 82159eb8aadb4973b65d7edc4f417d84 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250112 20:57:35.813688 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82159eb8aadb4973b65d7edc4f417d84. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.813921 16678 leader_election.cc:343] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.789326 16830 tablet_bootstrap.cc:492] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:35.816654 16771 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef14661bd7346f68101fe2a0fb064bb (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250112 20:57:35.818146 16771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ef14661bd7346f68101fe2a0fb064bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.832455 16777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f61fd1d8bf44f64b73ade11c0224df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.833550 16696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f61fd1d8bf44f64b73ade11c0224df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.845346 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:35.845583 16769 tablet_service.cc:1467] Processing CreateTablet for tablet 82159eb8aadb4973b65d7edc4f417d84 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250112 20:57:35.846009 16785 raft_consensus.cc:2463] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:35.846773 16769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82159eb8aadb4973b65d7edc4f417d84. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:35.848340 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:35.854427 16830 tablet_bootstrap.cc:654] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:35.865662 16828 ts_tablet_manager.cc:1428] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.295s user 0.008s sys 0.008s
I20250112 20:57:35.865303 16770 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0f7e20973149ed977de7cdd902fc39 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250112 20:57:35.866626 16828 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:35.866796 16770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0f7e20973149ed977de7cdd902fc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.880034 16680 leader_election.cc:304] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:35.880849 16862 raft_consensus.cc:2798] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:35.881223 16862 raft_consensus.cc:491] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:35.881520 16862 raft_consensus.cc:3054] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:35.832856 16697 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0f7e20973149ed977de7cdd902fc39 (DEFAULT_TABLE table=test-workload [id=7fe248d2bb09479ba0878b9c42545c37]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250112 20:57:35.889668 16697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a0f7e20973149ed977de7cdd902fc39. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:35.892253 16862 raft_consensus.cc:513] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:35.888543 16828 tablet_bootstrap.cc:654] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:35.904402 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
W20250112 20:57:35.905679 16678 leader_election.cc:343] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:35.906169 16862 leader_election.cc:290] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713), 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:35.907503 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:35.908103 16785 raft_consensus.cc:3054] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:35.914297 16785 raft_consensus.cc:2463] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:35.915441 16680 leader_election.cc:304] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:35.926879 16862 raft_consensus.cc:2798] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:35.937794 16829 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:35.938258 16829 ts_tablet_manager.cc:1397] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.260s user 0.009s sys 0.020s
I20250112 20:57:35.939430 16853 consensus_queue.cc:1035] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:35.940915 16829 raft_consensus.cc:357] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.941596 16829 raft_consensus.cc:383] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:35.941874 16829 raft_consensus.cc:738] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:35.942553 16829 consensus_queue.cc:260] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:35.945200 16829 ts_tablet_manager.cc:1428] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20250112 20:57:35.928345 16862 raft_consensus.cc:695] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:35.946578 16862 consensus_queue.cc:237] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:35.975450 16829 tablet_bootstrap.cc:492] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
W20250112 20:57:35.980837 16751 consensus_peers.cc:487] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:35.983157 16830 tablet_bootstrap.cc:492] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:35.984508 16830 ts_tablet_manager.cc:1397] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.195s user 0.052s sys 0.072s
I20250112 20:57:36.000113 16829 tablet_bootstrap.cc:654] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:36.001660 16537 catalog_manager.cc:5526] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:36.009989 16830 raft_consensus.cc:357] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.012226 16830 raft_consensus.cc:383] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.012526 16830 raft_consensus.cc:738] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.013175 16830 consensus_queue.cc:260] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.015791 16830 ts_tablet_manager.cc:1428] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20250112 20:57:36.016788 16830 tablet_bootstrap.cc:492] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:36.022249 16828 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:36.022644 16830 tablet_bootstrap.cc:654] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:36.024149 16828 ts_tablet_manager.cc:1397] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.158s user 0.026s sys 0.016s
I20250112 20:57:36.024914 16829 tablet_bootstrap.cc:492] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:36.025345 16829 ts_tablet_manager.cc:1397] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.009s
I20250112 20:57:36.028131 16829 raft_consensus.cc:357] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.029105 16829 raft_consensus.cc:383] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.029477 16829 raft_consensus.cc:738] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.029629 16828 raft_consensus.cc:357] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.030473 16828 raft_consensus.cc:383] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.030303 16829 consensus_queue.cc:260] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.031092 16828 raft_consensus.cc:738] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.032047 16863 raft_consensus.cc:491] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:36.032650 16863 raft_consensus.cc:513] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.033984 16830 tablet_bootstrap.cc:492] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:36.034399 16830 ts_tablet_manager.cc:1397] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.003s
I20250112 20:57:36.035113 16863 leader_election.cc:290] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:36.036907 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:36.038419 16605 leader_election.cc:343] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:36.040102 16828 consensus_queue.cc:260] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.042572 16829 ts_tablet_manager.cc:1428] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.017s user 0.005s sys 0.010s
I20250112 20:57:36.043309 16828 ts_tablet_manager.cc:1428] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.019s user 0.003s sys 0.003s
I20250112 20:57:36.043500 16829 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:36.044232 16828 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:36.045481 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:36.049057 16602 leader_election.cc:343] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:36.049444 16602 leader_election.cc:304] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:36.049867 16829 tablet_bootstrap.cc:654] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:36.050396 16828 tablet_bootstrap.cc:654] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:36.050397 16863 raft_consensus.cc:2743] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:36.056823 16875 raft_consensus.cc:491] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:36.057235 16875 raft_consensus.cc:513] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.066812 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bf3756af284af8b3ec4d29bbf5b972" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:36.069537 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bf3756af284af8b3ec4d29bbf5b972" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:36.069397 16830 raft_consensus.cc:357] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.070003 16830 raft_consensus.cc:383] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.070084 16714 raft_consensus.cc:2463] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:36.070302 16830 raft_consensus.cc:738] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.070958 16830 consensus_queue.cc:260] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.071393 16751 leader_election.cc:304] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:36.072110 16853 raft_consensus.cc:2798] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:36.072422 16853 raft_consensus.cc:491] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:36.072713 16853 raft_consensus.cc:3054] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:36.068429 16875 leader_election.cc:290] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:36.076862 16875 raft_consensus.cc:491] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:36.077251 16875 raft_consensus.cc:513] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.078898 16875 leader_election.cc:290] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:36.080279 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:36.080871 16638 raft_consensus.cc:2463] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
W20250112 20:57:36.081655 16751 leader_election.cc:343] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:36.083343 16830 ts_tablet_manager.cc:1428] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.049s user 0.005s sys 0.002s
I20250112 20:57:36.085505 16853 raft_consensus.cc:513] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.088306 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bf3756af284af8b3ec4d29bbf5b972" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:36.088829 16714 raft_consensus.cc:3054] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:36.091935 16830 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:36.094796 16714 raft_consensus.cc:2463] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:36.101820 16853 leader_election.cc:290] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:36.103394 16751 leader_election.cc:304] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:36.104147 16853 raft_consensus.cc:2798] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:36.105319 16853 raft_consensus.cc:695] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:36.106173 16853 consensus_queue.cc:237] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.108662 16829 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:36.109144 16829 ts_tablet_manager.cc:1397] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.066s user 0.015s sys 0.004s
I20250112 20:57:36.109404 16828 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:36.109982 16828 ts_tablet_manager.cc:1397] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.066s user 0.026s sys 0.032s
I20250112 20:57:36.111864 16829 raft_consensus.cc:357] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.113431 16829 raft_consensus.cc:383] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.113795 16829 raft_consensus.cc:738] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.113270 16828 raft_consensus.cc:357] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.114271 16828 raft_consensus.cc:383] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.114620 16828 raft_consensus.cc:738] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.114543 16829 consensus_queue.cc:260] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.115778 16828 consensus_queue.cc:260] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.117324 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:36.117360 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90bf3756af284af8b3ec4d29bbf5b972" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:36.118063 16638 raft_consensus.cc:3054] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:36.119563 16830 tablet_bootstrap.cc:654] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:36.118032 16639 raft_consensus.cc:2463] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:36.131423 16752 leader_election.cc:304] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:36.124840 16638 raft_consensus.cc:2463] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:36.133745 16853 raft_consensus.cc:2798] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:36.134196 16853 raft_consensus.cc:491] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:36.134588 16853 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:36.134781 16535 catalog_manager.cc:5526] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:36.125326 16828 ts_tablet_manager.cc:1428] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.015s user 0.009s sys 0.000s
I20250112 20:57:36.139122 16828 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:36.149729 16853 raft_consensus.cc:513] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.153137 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:36.153836 16862 raft_consensus.cc:491] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:36.153927 16638 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:36.154332 16862 raft_consensus.cc:513] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.156219 16862 leader_election.cc:290] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:36.157595 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:36.158658 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:36.126286 16829 ts_tablet_manager.cc:1428] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.017s user 0.004s sys 0.002s
I20250112 20:57:36.160496 16678 leader_election.cc:304] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:36.160490 16638 raft_consensus.cc:2463] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:36.161381 16829 tablet_bootstrap.cc:492] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:36.161929 16752 leader_election.cc:304] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:36.162700 16862 raft_consensus.cc:2743] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:36.163491 16880 raft_consensus.cc:2798] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:36.163936 16880 raft_consensus.cc:695] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:36.164820 16828 tablet_bootstrap.cc:654] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:36.164728 16880 consensus_queue.cc:237] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.169078 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:36.170246 16829 tablet_bootstrap.cc:654] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:36.170486 16751 leader_election.cc:343] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:36.171224 16853 leader_election.cc:290] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:36.175393 16830 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:36.175818 16830 ts_tablet_manager.cc:1397] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.084s user 0.010s sys 0.018s
I20250112 20:57:36.177345 16828 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:36.177784 16828 ts_tablet_manager.cc:1397] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.003s
I20250112 20:57:36.178187 16830 raft_consensus.cc:357] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:36.178850 16830 raft_consensus.cc:383] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.179102 16830 raft_consensus.cc:738] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.179494 16829 tablet_bootstrap.cc:492] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:36.179980 16829 ts_tablet_manager.cc:1397] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.010s
I20250112 20:57:36.180348 16828 raft_consensus.cc:357] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.181114 16828 raft_consensus.cc:383] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.182021 16853 raft_consensus.cc:491] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:36.182492 16853 raft_consensus.cc:513] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.182363 16829 raft_consensus.cc:357] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.181555 16828 raft_consensus.cc:738] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.185442 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3eb6b1d35d4c23bb0732c8c18c56ab" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:36.187037 16714 raft_consensus.cc:2463] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:36.185521 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3eb6b1d35d4c23bb0732c8c18c56ab" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:36.183274 16829 raft_consensus.cc:383] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:36.189003 16751 leader_election.cc:304] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:36.189425 16829 raft_consensus.cc:738] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:36.190043 16873 raft_consensus.cc:2798] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:36.190430 16873 raft_consensus.cc:491] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:36.190001 16829 consensus_queue.cc:260] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.190716 16873 raft_consensus.cc:3054] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:36.192371 16829 ts_tablet_manager.cc:1428] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
W20250112 20:57:36.192730 16752 leader_election.cc:343] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:36.193320 16829 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:36.194769 16830 consensus_queue.cc:260] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:36.185560 16853 leader_election.cc:290] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:36.198685 16829 tablet_bootstrap.cc:654] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:36.198499 16873 raft_consensus.cc:513] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.201117 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3eb6b1d35d4c23bb0732c8c18c56ab" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:36.201633 16714 raft_consensus.cc:3054] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:36.205744 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3eb6b1d35d4c23bb0732c8c18c56ab" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
W20250112 20:57:36.207122 16752 leader_election.cc:343] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:36.207839 16714 raft_consensus.cc:2463] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:36.208271 16830 ts_tablet_manager.cc:1428] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.032s user 0.005s sys 0.002s
I20250112 20:57:36.208849 16751 leader_election.cc:304] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:36.209100 16830 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:36.210357 16853 raft_consensus.cc:2798] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:36.210922 16853 raft_consensus.cc:695] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:36.211840 16853 consensus_queue.cc:237] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.212442 16829 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:36.212908 16829 ts_tablet_manager.cc:1397] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.004s
I20250112 20:57:36.186206 16828 consensus_queue.cc:260] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:36.215245 16873 leader_election.cc:290] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:36.217389 16830 tablet_bootstrap.cc:654] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:36.218286 16828 ts_tablet_manager.cc:1428] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.040s user 0.003s sys 0.004s
I20250112 20:57:36.219424 16828 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:37.302788 16828 tablet_bootstrap.cc:654] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.309643 16828 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:37.310004 16828 ts_tablet_manager.cc:1397] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 1.091s user 0.016s sys 0.000s
I20250112 20:57:37.312635 16828 raft_consensus.cc:357] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.313194 16828 raft_consensus.cc:383] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.313397 16828 raft_consensus.cc:738] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.314052 16828 consensus_queue.cc:260] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.316094 16828 ts_tablet_manager.cc:1428] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:57:37.316933 16828 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:37.322355 16828 tablet_bootstrap.cc:654] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.331388 16828 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:37.331789 16828 ts_tablet_manager.cc:1397] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20250112 20:57:37.334079 16828 raft_consensus.cc:357] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.334769 16828 raft_consensus.cc:383] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.335031 16828 raft_consensus.cc:738] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.335590 16828 consensus_queue.cc:260] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.338469 16891 raft_consensus.cc:491] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.338902 16891 raft_consensus.cc:513] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.339854 16536 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50356 (request call id 16) took 1141 ms. Trace:
I20250112 20:57:37.340289 16536 rpcz_store.cc:276] 0112 20:57:36.198931 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:57:36.199369 (+ 438us) service_pool.cc:225] Handling call
0112 20:57:37.339784 (+1140415us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
W20250112 20:57:37.343822 16568 debug-util.cc:398] Leaking SignalData structure 0x7b080008f8e0 after lost signal to thread 15457
I20250112 20:57:37.345764 16828 ts_tablet_manager.cc:1428] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.014s user 0.010s sys 0.000s
I20250112 20:57:37.346673 16828 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
W20250112 20:57:37.349738 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:37.351084 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153749bb76c146f2ba4e7d07efbc1497" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.352411 16828 tablet_bootstrap.cc:654] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:37.352650 16602 leader_election.cc:343] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.352807 16892 raft_consensus.cc:491] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.353874 16892 raft_consensus.cc:513] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.355918 16892 leader_election.cc:290] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.353158 16891 leader_election.cc:290] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.356561 16891 raft_consensus.cc:491] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.357000 16891 raft_consensus.cc:513] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } }
I20250112 20:57:37.357975 16535 rpcz_store.cc:275] Call kudu.master.MasterService.IsCreateTableDone from 127.0.0.1:50374 (request call id 29) took 1014 ms. Trace:
I20250112 20:57:37.358371 16535 rpcz_store.cc:276] 0112 20:57:36.343622 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:57:37.357174 (+1013552us) service_pool.cc:225] Handling call
0112 20:57:37.357559 (+ 385us) catalog_manager.cc:2348] Looking up, locking, and authorizing table
0112 20:57:37.357747 (+ 188us) catalog_manager.cc:2246] Verify if the table creation is in progress for test-workload [id=7fe248d2bb09479ba0878b9c42545c37]
0112 20:57:37.357918 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:57:37.358843 16893 raft_consensus.cc:491] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.359772 16891 leader_election.cc:290] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:37.359970 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.359406 16893 raft_consensus.cc:513] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.366772 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a2acedfaa2476596c2ec8bb689dcce" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.367558 16714 raft_consensus.cc:2463] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:37.368633 16602 leader_election.cc:304] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:37.370119 16891 raft_consensus.cc:2798] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:37.372907 16891 raft_consensus.cc:491] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:37.373219 16891 raft_consensus.cc:3054] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.378922 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abd904e2af24417f8b366444ad932e30" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:37.379228 16828 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:37.379668 16828 ts_tablet_manager.cc:1397] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.011s
I20250112 20:57:37.386740 16863 raft_consensus.cc:491] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.387303 16863 raft_consensus.cc:513] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.389665 16863 leader_election.cc:290] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.391006 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:37.393656 16602 leader_election.cc:343] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:37.396934 16678 leader_election.cc:343] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.397241 16891 raft_consensus.cc:513] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.378649 16893 leader_election.cc:290] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.399693 16891 leader_election.cc:290] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.400382 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a2acedfaa2476596c2ec8bb689dcce" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:37.401356 16828 raft_consensus.cc:357] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.401990 16714 raft_consensus.cc:3054] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.407665 16638 raft_consensus.cc:3054] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.409042 16714 raft_consensus.cc:2463] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:37.402009 16828 raft_consensus.cc:383] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.410094 16828 raft_consensus.cc:738] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.410786 16828 consensus_queue.cc:260] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.412951 16828 ts_tablet_manager.cc:1428] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.033s user 0.008s sys 0.000s
I20250112 20:57:37.414037 16828 tablet_bootstrap.cc:492] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:37.415696 16602 leader_election.cc:304] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:37.416491 16891 raft_consensus.cc:2798] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:37.416498 16894 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:37.417060 16891 raft_consensus.cc:695] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:37.417815 16891 consensus_queue.cc:237] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.419498 16828 tablet_bootstrap.cc:654] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:37.412927 16830 thread.cc:635] log-alloc [worker] (thread pool) Time spent creating pthread: real 1.191s user 0.009s sys 0.008s
W20250112 20:57:37.420853 16830 thread.cc:601] log-alloc [worker] (thread pool) Time spent starting thread: real 1.199s user 0.009s sys 0.008s
I20250112 20:57:37.422986 16830 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:37.423446 16830 ts_tablet_manager.cc:1397] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 1.215s user 0.018s sys 0.009s
I20250112 20:57:37.426671 16830 raft_consensus.cc:357] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.429134 16830 raft_consensus.cc:383] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.429783 16830 raft_consensus.cc:738] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.430044 16828 tablet_bootstrap.cc:492] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:37.433362 16828 ts_tablet_manager.cc:1397] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.008s
I20250112 20:57:37.432936 16830 consensus_queue.cc:260] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.435631 16828 raft_consensus.cc:357] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.436332 16828 raft_consensus.cc:383] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.436641 16828 raft_consensus.cc:738] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.437287 16828 consensus_queue.cc:260] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.437788 16830 ts_tablet_manager.cc:1428] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.014s user 0.005s sys 0.001s
I20250112 20:57:37.439911 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153749bb76c146f2ba4e7d07efbc1497" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.440121 16828 ts_tablet_manager.cc:1428] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:57:37.440263 16830 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:37.445451 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a2acedfaa2476596c2ec8bb689dcce" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.436192 16829 raft_consensus.cc:357] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.447317 16829 raft_consensus.cc:383] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.447599 16829 raft_consensus.cc:738] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
W20250112 20:57:37.450584 16605 leader_election.cc:343] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.441002 16828 tablet_bootstrap.cc:492] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:37.441200 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.451020 16605 leader_election.cc:304] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.451526 16786 raft_consensus.cc:2388] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80770068c3a34f918765f54dbe3a94eb in current term 1: Already voted for candidate 018d03f043b542b9b63491a77523968d in this term.
I20250112 20:57:37.441951 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.442827 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a2acedfaa2476596c2ec8bb689dcce" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:37.453830 16830 tablet_bootstrap.cc:654] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:37.456152 16568 debug-util.cc:398] Leaking SignalData structure 0x7b080003d000 after lost signal to thread 16571
W20250112 20:57:37.457440 16568 debug-util.cc:398] Leaking SignalData structure 0x7b0800177de0 after lost signal to thread 16663
W20250112 20:57:37.458698 16568 debug-util.cc:398] Leaking SignalData structure 0x7b08000bfa40 after lost signal to thread 16738
I20250112 20:57:37.443315 16873 raft_consensus.cc:491] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.460063 16873 raft_consensus.cc:513] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.462299 16873 leader_election.cc:290] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:37.467097 16828 tablet_bootstrap.cc:654] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.448192 16829 consensus_queue.cc:260] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.472985 16829 ts_tablet_manager.cc:1428] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 1.251s user 0.009s sys 0.000s
W20250112 20:57:37.473343 16829 ts_tablet_manager.cc:1460] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Tablet startup took 1260ms
I20250112 20:57:37.491991 16863 raft_consensus.cc:2743] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:37.492352 16568 debug-util.cc:398] Leaking SignalData structure 0x7b08001ae320 after lost signal to thread 16812
W20250112 20:57:37.494491 16568 debug-util.cc:398] Leaking SignalData structure 0x7b08000cf1c0 after lost signal to thread 16845
W20250112 20:57:37.496969 16605 leader_election.cc:343] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.497978 16605 leader_election.cc:304] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
W20250112 20:57:37.502213 16605 leader_election.cc:343] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.502702 16605 leader_election.cc:304] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.503479 16891 raft_consensus.cc:2743] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:37.505142 16891 raft_consensus.cc:491] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.505797 16891 raft_consensus.cc:513] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.508527 16891 leader_election.cc:290] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:37.504693 16605 leader_election.cc:343] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.509351 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:37.510589 16602 leader_election.cc:343] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.510556 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:37.512303 16605 leader_election.cc:343] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.512759 16605 leader_election.cc:304] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.513868 16891 raft_consensus.cc:2743] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:37.476692 16829 ts_tablet_manager.cc:1462] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Trace:
0112 20:57:34.382745 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:57:34.582913 (+200168us) service_pool.cc:225] Handling call
0112 20:57:34.589352 (+ 6439us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:57:34.589453 (+ 101us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:57:34.701645 (+112192us) tablet_metadata.cc:641] Metadata flushed
0112 20:57:34.708323 (+ 6678us) tablet_replica.cc:206] Creating consensus instance
0112 20:57:34.708784 (+ 461us) inbound_call.cc:173] Queueing success response
0112 20:57:36.193016 (+1484232us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0112 20:57:36.222453 (+ 29437us) ts_tablet_manager.cc:1429] Starting tablet replica
0112 20:57:36.334838 (+112385us) tablet_replica.cc:260] Starting instrumentation
0112 20:57:36.335400 (+ 562us) tablet_replica.cc:279] Starting consensus
Metrics: {"mutex_wait_us":1093322,"raft.queue_time_us":100,"tablet-open.queue_time_us":1484347}
I20250112 20:57:37.518059 16829 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:37.525990 16863 raft_consensus.cc:2743] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:37.526857 16714 raft_consensus.cc:3054] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.529292 16830 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:37.529753 16830 ts_tablet_manager.cc:1397] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.090s user 0.016s sys 0.014s
I20250112 20:57:37.532142 16830 raft_consensus.cc:357] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.532934 16830 raft_consensus.cc:383] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.533244 16830 raft_consensus.cc:738] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.533939 16830 consensus_queue.cc:260] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.535403 16891 raft_consensus.cc:491] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c29b243acf2b4f248606563fc96efc7d)
I20250112 20:57:37.535869 16891 raft_consensus.cc:513] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.536756 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e510cf639924856a759b20574cb60be" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.537380 16713 raft_consensus.cc:2463] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:37.537806 16891 leader_election.cc:290] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.540088 16829 tablet_bootstrap.cc:654] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.540030 16751 leader_election.cc:304] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:37.540596 16830 ts_tablet_manager.cc:1428] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250112 20:57:37.541100 16900 raft_consensus.cc:2798] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:37.541577 16830 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
W20250112 20:57:37.542027 16751 consensus_peers.cc:487] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:37.542532 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.543617 16863 raft_consensus.cc:491] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.544035 16863 raft_consensus.cc:513] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.546268 16863 leader_election.cc:290] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.547765 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abd904e2af24417f8b366444ad932e30" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.548280 16830 tablet_bootstrap.cc:654] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.548410 16712 raft_consensus.cc:2463] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:37.549615 16602 leader_election.cc:304] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:37.541577 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0f35368d307429d8a87ba0c17bb4caf" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.541576 16900 raft_consensus.cc:491] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:37.550954 16900 raft_consensus.cc:3054] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.552330 16863 raft_consensus.cc:2798] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:37.552812 16863 raft_consensus.cc:491] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:37.553210 16863 raft_consensus.cc:3054] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.553653 16891 raft_consensus.cc:491] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.554093 16891 raft_consensus.cc:513] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.555936 16891 leader_election.cc:290] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.557919 16605 leader_election.cc:304] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.559624 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.561038 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abd904e2af24417f8b366444ad932e30" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.539764 16535 catalog_manager.cc:5526] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:37.577733 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.582788 16863 raft_consensus.cc:513] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.565569 16891 raft_consensus.cc:2743] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:57:37.592614 16863 leader_election.cc:290] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:37.588496 16605 leader_election.cc:343] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.567449 16900 raft_consensus.cc:513] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.594583 16899 raft_consensus.cc:491] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.596340 16899 raft_consensus.cc:513] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.598197 16880 consensus_queue.cc:1035] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:37.598645 16605 leader_election.cc:343] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.599088 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abd904e2af24417f8b366444ad932e30" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
W20250112 20:57:37.595587 16602 leader_election.cc:343] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:37.600478 16605 leader_election.cc:343] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.599215 16899 leader_election.cc:290] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:37.601504 16602 leader_election.cc:304] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.602350 16891 raft_consensus.cc:2743] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:37.601581 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.591269 16873 raft_consensus.cc:491] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.606271 16873 raft_consensus.cc:513] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.606743 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abd904e2af24417f8b366444ad932e30" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:37.608713 16711 raft_consensus.cc:3054] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.609737 16873 leader_election.cc:290] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:37.612069 16828 tablet_bootstrap.cc:492] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:37.612679 16828 ts_tablet_manager.cc:1397] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.172s user 0.020s sys 0.009s
I20250112 20:57:37.617579 16900 leader_election.cc:290] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:37.622220 16830 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:37.622602 16830 ts_tablet_manager.cc:1397] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.081s user 0.017s sys 0.021s
I20250112 20:57:37.605798 16713 raft_consensus.cc:2463] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:37.613835 16710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3f67e9a10d448f93128a7f95344801" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.635154 16710 raft_consensus.cc:2463] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:37.634043 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abd904e2af24417f8b366444ad932e30" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.638795 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e510cf639924856a759b20574cb60be" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:37.641443 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:37.657379 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3f67e9a10d448f93128a7f95344801" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:37.683523 16711 raft_consensus.cc:2463] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:37.684789 16602 leader_election.cc:304] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.686103 16891 raft_consensus.cc:2798] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:37.686605 16891 raft_consensus.cc:695] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:37.687520 16891 consensus_queue.cc:237] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.691684 16829 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:37.691684 16830 raft_consensus.cc:357] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.692915 16829 ts_tablet_manager.cc:1397] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.175s user 0.028s sys 0.088s
I20250112 20:57:37.692831 16830 raft_consensus.cc:383] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.693346 16830 raft_consensus.cc:738] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.693996 16830 consensus_queue.cc:260] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.695849 16830 ts_tablet_manager.cc:1428] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.073s user 0.012s sys 0.020s
I20250112 20:57:37.696837 16830 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:37.696264 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e510cf639924856a759b20574cb60be" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:37.697233 16900 raft_consensus.cc:491] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.697644 16873 consensus_queue.cc:1035] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:37.697803 16752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:41463, user_credentials={real_user=slave}} blocked reactor thread for 64103.4us
I20250112 20:57:37.697664 16900 raft_consensus.cc:513] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:37.699148 16680 leader_election.cc:343] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:37.699379 16752 consensus_peers.cc:487] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:37.699611 16680 leader_election.cc:304] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.701388 16862 raft_consensus.cc:2743] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:37.701968 16873 consensus_queue.cc:1035] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250112 20:57:37.702888 16752 leader_election.cc:343] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.701506 16829 raft_consensus.cc:357] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.703346 16829 raft_consensus.cc:383] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.701503 16892 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250112 20:57:37.703629 16829 raft_consensus.cc:738] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
W20250112 20:57:37.703786 16752 leader_election.cc:343] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:37.704348 16752 leader_election.cc:343] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.704290 16829 consensus_queue.cc:260] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.701329 16900 leader_election.cc:290] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:37.706830 16829 ts_tablet_manager.cc:1428] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250112 20:57:37.706696 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e510cf639924856a759b20574cb60be" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:37.707603 16829 tablet_bootstrap.cc:492] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:37.707753 16712 raft_consensus.cc:3054] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.713934 16828 raft_consensus.cc:357] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.714411 16828 raft_consensus.cc:383] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.714681 16828 raft_consensus.cc:738] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
W20250112 20:57:37.705020 16752 leader_election.cc:343] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.716327 16712 raft_consensus.cc:2463] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:37.716993 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:37.715277 16828 consensus_queue.cc:260] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:37.721660 16752 leader_election.cc:343] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.722404 16828 ts_tablet_manager.cc:1428] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.109s user 0.007s sys 0.012s
I20250112 20:57:37.724022 16828 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:37.726032 16829 tablet_bootstrap.cc:654] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.727170 16830 tablet_bootstrap.cc:654] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.734378 16828 tablet_bootstrap.cc:654] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.727734 16751 leader_election.cc:304] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:37.732043 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:37.741828 16875 raft_consensus.cc:2798] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:37.742215 16875 raft_consensus.cc:491] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:37.742555 16875 raft_consensus.cc:3054] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.750409 16875 raft_consensus.cc:513] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.752252 16875 leader_election.cc:290] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:37.765374 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:37.764925 16751 leader_election.cc:304] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
W20250112 20:57:37.770009 16752 leader_election.cc:343] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.770738 16875 raft_consensus.cc:2798] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:37.771198 16875 raft_consensus.cc:491] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:37.771585 16875 raft_consensus.cc:3054] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.775274 16900 consensus_queue.cc:1035] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:37.775643 16751 leader_election.cc:304] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:37.776576 16900 raft_consensus.cc:2798] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
W20250112 20:57:37.776887 16751 leader_election.cc:343] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.777096 16900 raft_consensus.cc:695] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:37.777362 16751 leader_election.cc:304] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:37.777917 16900 consensus_queue.cc:237] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.778734 16899 raft_consensus.cc:2743] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:37.785415 16892 raft_consensus.cc:491] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.786052 16892 raft_consensus.cc:513] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.789846 16862 raft_consensus.cc:491] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.790464 16862 raft_consensus.cc:513] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.792320 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:37.793107 16712 raft_consensus.cc:3054] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.812697 16788 raft_consensus.cc:3054] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.822677 16894 raft_consensus.cc:491] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:37.823202 16894 raft_consensus.cc:513] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.827041 16894 leader_election.cc:290] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.827687 16875 raft_consensus.cc:513] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.828755 16892 leader_election.cc:290] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.831992 16830 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:37.832326 16830 ts_tablet_manager.cc:1397] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.136s user 0.012s sys 0.002s
I20250112 20:57:37.836850 16828 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:37.837258 16828 ts_tablet_manager.cc:1397] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.113s user 0.012s sys 0.007s
I20250112 20:57:37.834486 16862 leader_election.cc:290] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.853782 16829 tablet_bootstrap.cc:492] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:37.869336 16830 raft_consensus.cc:357] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.870177 16829 ts_tablet_manager.cc:1397] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.163s user 0.015s sys 0.004s
I20250112 20:57:37.870216 16830 raft_consensus.cc:383] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.870631 16830 raft_consensus.cc:738] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.851382 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:37.870975 16891 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:37.871357 16638 raft_consensus.cc:2463] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:37.871325 16830 consensus_queue.cc:260] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.871961 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:37.873029 16680 leader_election.cc:343] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.873994 16712 raft_consensus.cc:2463] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:37.874534 16863 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:37.875118 16751 leader_election.cc:304] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:37.846169 16875 leader_election.cc:290] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:37.876438 16875 raft_consensus.cc:2798] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:37.876848 16875 raft_consensus.cc:695] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:37.877118 16830 ts_tablet_manager.cc:1428] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.045s user 0.006s sys 0.001s
I20250112 20:57:37.878113 16830 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:37.880306 16828 raft_consensus.cc:357] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.880553 16678 leader_election.cc:304] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.880956 16828 raft_consensus.cc:383] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:37.881249 16828 raft_consensus.cc:738] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.885048 16892 raft_consensus.cc:2798] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:37.851822 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3f67e9a10d448f93128a7f95344801" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:37.851680 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3eb6b1d35d4c23bb0732c8c18c56ab" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:37.852839 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153749bb76c146f2ba4e7d07efbc1497" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:37.885421 16892 raft_consensus.cc:491] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:37.887864 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153749bb76c146f2ba4e7d07efbc1497" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:37.889710 16892 raft_consensus.cc:3054] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.889606 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3f67e9a10d448f93128a7f95344801" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:37.890159 16637 raft_consensus.cc:2463] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
W20250112 20:57:37.891413 16752 leader_election.cc:343] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.891534 16678 leader_election.cc:304] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:37.892804 16635 raft_consensus.cc:3054] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.896761 16894 raft_consensus.cc:2798] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:37.897111 16894 raft_consensus.cc:491] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:37.897421 16894 raft_consensus.cc:3054] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.852228 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3eb6b1d35d4c23bb0732c8c18c56ab" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:37.907048 16680 leader_election.cc:343] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:37.908740 16680 leader_election.cc:304] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:37.913446 16861 raft_consensus.cc:2743] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:57:37.885319 16875 consensus_queue.cc:237] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.911432 16714 raft_consensus.cc:3054] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.921077 16892 raft_consensus.cc:513] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.953311 16828 consensus_queue.cc:260] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.954288 16829 raft_consensus.cc:357] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.957634 16829 raft_consensus.cc:383] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20250112 20:57:37.956919 16923 log.cc:926] Time spent T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
I20250112 20:57:37.956295 16828 ts_tablet_manager.cc:1428] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.119s user 0.035s sys 0.033s
I20250112 20:57:37.951910 16894 raft_consensus.cc:513] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.960157 16829 raft_consensus.cc:738] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:37.960870 16829 consensus_queue.cc:260] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.959896 16828 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:37.962256 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153749bb76c146f2ba4e7d07efbc1497" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:37.962810 16636 raft_consensus.cc:3054] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.966554 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:37.967379 16638 raft_consensus.cc:3054] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:37.974195 16830 tablet_bootstrap.cc:654] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.975454 16638 raft_consensus.cc:2463] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:37.976620 16678 leader_election.cc:304] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:37.977389 16861 raft_consensus.cc:2798] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:37.977830 16829 ts_tablet_manager.cc:1428] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.107s user 0.013s sys 0.013s
I20250112 20:57:37.977792 16861 raft_consensus.cc:695] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:37.978698 16829 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:37.978585 16861 consensus_queue.cc:237] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:37.995967 16894 leader_election.cc:290] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.041049 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153749bb76c146f2ba4e7d07efbc1497" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:38.049613 16787 raft_consensus.cc:3054] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.045074 16536 catalog_manager.cc:5526] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:38.050311 16828 tablet_bootstrap.cc:654] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:37.994874 16892 leader_election.cc:290] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:37.985183 16636 raft_consensus.cc:2463] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
W20250112 20:57:38.051714 16897 log.cc:926] Time spent T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: Append to log took a long time: real 0.063s user 0.007s sys 0.003s
I20250112 20:57:38.055244 16678 leader_election.cc:304] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:38.068333 16829 tablet_bootstrap.cc:654] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.069962 16861 raft_consensus.cc:2798] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:38.016223 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:38.070621 16861 raft_consensus.cc:695] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:38.068094 16537 catalog_manager.cc:5526] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:38.072361 16537 catalog_manager.cc:5526] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } }
I20250112 20:57:38.073442 16861 consensus_queue.cc:237] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.077598 16863 raft_consensus.cc:491] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.077971 16787 raft_consensus.cc:2463] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:38.078135 16863 raft_consensus.cc:513] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.082721 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b544e7e4fa4e419949c833428c406b" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:38.084244 16602 leader_election.cc:343] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.080367 16863 leader_election.cc:290] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.086123 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b544e7e4fa4e419949c833428c406b" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:38.087638 16605 leader_election.cc:343] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:38.087890 16680 leader_election.cc:343] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.088045 16605 leader_election.cc:304] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:38.060906 16714 raft_consensus.cc:2463] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:38.089160 16863 raft_consensus.cc:2743] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:38.093986 16830 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:38.094306 16830 ts_tablet_manager.cc:1397] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.217s user 0.014s sys 0.005s
I20250112 20:57:38.092049 16751 leader_election.cc:304] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:38.096849 16875 raft_consensus.cc:2798] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:38.097285 16875 raft_consensus.cc:695] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:38.097986 16875 consensus_queue.cc:237] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.101745 16829 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:38.102154 16829 ts_tablet_manager.cc:1397] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.124s user 0.016s sys 0.012s
I20250112 20:57:38.106143 16829 raft_consensus.cc:357] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.106849 16829 raft_consensus.cc:383] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.107095 16829 raft_consensus.cc:738] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.111043 16829 consensus_queue.cc:260] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.108736 16830 raft_consensus.cc:357] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.113005 16829 ts_tablet_manager.cc:1428] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20250112 20:57:38.113032 16830 raft_consensus.cc:383] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.113494 16830 raft_consensus.cc:738] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.113925 16829 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:38.114071 16830 consensus_queue.cc:260] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.106101 16535 catalog_manager.cc:5526] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:38.119524 16830 ts_tablet_manager.cc:1428] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.025s user 0.005s sys 0.001s
I20250112 20:57:38.120293 16830 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:38.126119 16829 tablet_bootstrap.cc:654] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.130717 16828 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:38.130225 16536 catalog_manager.cc:5526] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:38.131153 16828 ts_tablet_manager.cc:1397] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.171s user 0.004s sys 0.021s
I20250112 20:57:38.131460 16892 raft_consensus.cc:491] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.131882 16892 raft_consensus.cc:513] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.133040 16830 tablet_bootstrap.cc:654] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.133440 16828 raft_consensus.cc:357] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.133706 16892 leader_election.cc:290] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.134076 16828 raft_consensus.cc:383] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.134424 16828 raft_consensus.cc:738] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.135100 16828 consensus_queue.cc:260] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.135815 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:38.136907 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:38.137398 16828 ts_tablet_manager.cc:1428] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20250112 20:57:38.138209 16828 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
W20250112 20:57:38.138142 16680 leader_election.cc:343] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.136458 16635 raft_consensus.cc:2463] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:38.139730 16678 leader_election.cc:304] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:38.140483 16892 raft_consensus.cc:2798] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:38.140841 16892 raft_consensus.cc:491] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:38.141137 16892 raft_consensus.cc:3054] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.142827 16536 catalog_manager.cc:5526] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:38.147972 16892 raft_consensus.cc:513] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.157362 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:38.157896 16635 raft_consensus.cc:3054] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.150277 16829 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:38.164906 16829 ts_tablet_manager.cc:1397] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.051s user 0.012s sys 0.000s
I20250112 20:57:38.155174 16828 tablet_bootstrap.cc:654] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.166800 16635 raft_consensus.cc:2463] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:38.171351 16678 leader_election.cc:304] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:38.172411 16830 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:38.172824 16830 ts_tablet_manager.cc:1397] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.053s user 0.021s sys 0.003s
I20250112 20:57:38.175065 16863 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:38.175848 16892 leader_election.cc:290] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.175184 16830 raft_consensus.cc:357] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.177302 16830 raft_consensus.cc:383] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.177570 16830 raft_consensus.cc:738] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.177654 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:38.177803 16829 raft_consensus.cc:357] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.178457 16829 raft_consensus.cc:383] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.178730 16829 raft_consensus.cc:738] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
W20250112 20:57:38.179664 16680 leader_election.cc:343] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.180163 16830 consensus_queue.cc:260] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.178910 16861 raft_consensus.cc:2798] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:38.181725 16861 raft_consensus.cc:695] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:38.182415 16861 consensus_queue.cc:237] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.183393 16830 ts_tablet_manager.cc:1428] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20250112 20:57:38.184233 16830 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:38.180382 16829 consensus_queue.cc:260] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.187054 16829 ts_tablet_manager.cc:1428] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20250112 20:57:38.187875 16829 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:38.188617 16537 catalog_manager.cc:5526] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:38.198160 16635 raft_consensus.cc:3054] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.202879 16875 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:38.208535 16828 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:38.209017 16828 ts_tablet_manager.cc:1397] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.071s user 0.004s sys 0.013s
I20250112 20:57:38.218902 16830 tablet_bootstrap.cc:654] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.219482 16829 tablet_bootstrap.cc:654] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.227137 16875 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:38.240371 16537 catalog_manager.cc:5526] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:38.255609 16787 raft_consensus.cc:3054] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.278149 16828 raft_consensus.cc:357] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.279315 16828 raft_consensus.cc:383] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.279598 16828 raft_consensus.cc:738] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.280135 16828 consensus_queue.cc:260] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.282109 16828 ts_tablet_manager.cc:1428] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.073s user 0.005s sys 0.000s
I20250112 20:57:38.287992 16828 tablet_bootstrap.cc:492] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:38.291314 16863 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:38.295225 16830 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:38.295894 16830 ts_tablet_manager.cc:1397] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.112s user 0.016s sys 0.035s
I20250112 20:57:38.305707 16830 raft_consensus.cc:357] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.306427 16830 raft_consensus.cc:383] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.306691 16830 raft_consensus.cc:738] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.307327 16830 consensus_queue.cc:260] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.309145 16537 catalog_manager.cc:5526] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:38.310148 16830 ts_tablet_manager.cc:1428] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250112 20:57:38.310367 16829 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:38.312682 16829 ts_tablet_manager.cc:1397] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.125s user 0.013s sys 0.005s
I20250112 20:57:38.313612 16863 raft_consensus.cc:491] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.314375 16863 raft_consensus.cc:513] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.317174 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:38.317853 16711 raft_consensus.cc:2388] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80770068c3a34f918765f54dbe3a94eb in current term 1: Already voted for candidate c29b243acf2b4f248606563fc96efc7d in this term.
I20250112 20:57:38.319377 16830 tablet_bootstrap.cc:492] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:38.316514 16863 leader_election.cc:290] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.320261 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:38.321363 16605 leader_election.cc:304] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:38.335161 16828 tablet_bootstrap.cc:654] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.336723 16947 raft_consensus.cc:3054] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.339308 16829 raft_consensus.cc:357] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.339879 16829 raft_consensus.cc:383] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.340173 16829 raft_consensus.cc:738] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.341620 16829 consensus_queue.cc:260] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.345033 16829 ts_tablet_manager.cc:1428] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20250112 20:57:38.345672 16829 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:38.354993 16828 tablet_bootstrap.cc:492] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:38.355450 16828 ts_tablet_manager.cc:1397] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.068s user 0.011s sys 0.000s
I20250112 20:57:38.358968 16829 tablet_bootstrap.cc:654] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.359089 16828 raft_consensus.cc:357] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.359771 16828 raft_consensus.cc:383] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.359989 16828 raft_consensus.cc:738] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.360738 16828 consensus_queue.cc:260] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.364032 16828 ts_tablet_manager.cc:1428] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250112 20:57:38.365022 16828 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:38.369616 16947 raft_consensus.cc:2743] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:38.372848 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:38.384088 16830 tablet_bootstrap.cc:654] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.384460 16828 tablet_bootstrap.cc:654] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.385417 16829 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:38.388249 16829 ts_tablet_manager.cc:1397] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.043s user 0.014s sys 0.009s
I20250112 20:57:38.391491 16925 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:38.392525 16853 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:38.404460 16680 consensus_peers.cc:487] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:38.404573 16829 raft_consensus.cc:357] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.405280 16829 raft_consensus.cc:383] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.405596 16829 raft_consensus.cc:738] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.410339 16894 raft_consensus.cc:491] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.410969 16894 raft_consensus.cc:513] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.413175 16894 leader_election.cc:290] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.414662 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:38.415314 16636 raft_consensus.cc:2463] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:38.415549 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:38.416534 16678 leader_election.cc:304] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
W20250112 20:57:38.417126 16680 leader_election.cc:343] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.416833 16829 consensus_queue.cc:260] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.417375 16894 raft_consensus.cc:2798] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:38.417771 16894 raft_consensus.cc:491] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:38.418071 16894 raft_consensus.cc:3054] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.428507 16829 ts_tablet_manager.cc:1428] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20250112 20:57:38.437347 16830 tablet_bootstrap.cc:492] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:38.437844 16830 ts_tablet_manager.cc:1397] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.119s user 0.016s sys 0.002s
I20250112 20:57:38.440977 16830 raft_consensus.cc:357] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.441674 16830 raft_consensus.cc:383] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.441967 16830 raft_consensus.cc:738] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.442119 16894 raft_consensus.cc:513] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.442668 16830 consensus_queue.cc:260] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.445326 16830 ts_tablet_manager.cc:1428] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20250112 20:57:38.446457 16830 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:38.451550 16894 leader_election.cc:290] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.452448 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
W20250112 20:57:38.453796 16680 leader_election.cc:343] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.459739 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:38.461225 16925 raft_consensus.cc:491] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.461726 16925 raft_consensus.cc:513] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.462819 16829 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:38.467213 16925 leader_election.cc:290] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:38.465628 16635 raft_consensus.cc:3054] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.468324 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963bd285f345421581941f10dbe8b90c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:38.468396 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963bd285f345421581941f10dbe8b90c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:38.469077 16711 raft_consensus.cc:2463] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:38.469895 16828 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:38.470299 16828 ts_tablet_manager.cc:1397] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.105s user 0.015s sys 0.002s
I20250112 20:57:38.470135 16751 leader_election.cc:304] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:38.471503 16925 raft_consensus.cc:2798] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:38.472201 16925 raft_consensus.cc:491] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20250112 20:57:38.471351 16752 leader_election.cc:343] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.471355 16894 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:38.476984 16925 raft_consensus.cc:3054] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.481873 16828 raft_consensus.cc:357] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.482628 16828 raft_consensus.cc:383] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.482926 16828 raft_consensus.cc:738] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.483553 16828 consensus_queue.cc:260] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.485852 16828 ts_tablet_manager.cc:1428] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20250112 20:57:38.497363 16828 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:38.497853 16635 raft_consensus.cc:2463] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:38.498881 16678 leader_election.cc:304] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:38.500787 16894 raft_consensus.cc:2798] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:38.501195 16894 raft_consensus.cc:695] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:38.501917 16894 consensus_queue.cc:237] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.505051 16829 tablet_bootstrap.cc:654] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.509408 16711 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.510689 16828 tablet_bootstrap.cc:654] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.511986 16891 raft_consensus.cc:491] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.512476 16891 raft_consensus.cc:513] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.514328 16891 leader_election.cc:290] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.515153 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3f67e9a10d448f93128a7f95344801" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:38.515818 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a3f67e9a10d448f93128a7f95344801" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:38.499984 16925 raft_consensus.cc:513] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.516000 16714 raft_consensus.cc:2388] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80770068c3a34f918765f54dbe3a94eb in current term 1: Already voted for candidate c29b243acf2b4f248606563fc96efc7d in this term.
I20250112 20:57:38.523007 16602 leader_election.cc:304] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:38.524413 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963bd285f345421581941f10dbe8b90c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:38.524888 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963bd285f345421581941f10dbe8b90c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:38.525075 16714 raft_consensus.cc:3054] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.523276 16830 tablet_bootstrap.cc:654] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.526463 16891 raft_consensus.cc:3054] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:57:38.526611 16752 leader_election.cc:343] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.526072 16925 leader_election.cc:290] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:38.548555 16536 catalog_manager.cc:5526] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:38.551420 16714 raft_consensus.cc:2463] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:38.552552 16751 leader_election.cc:304] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:38.553934 16891 raft_consensus.cc:2743] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:38.557219 16925 raft_consensus.cc:2798] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:38.576424 16925 raft_consensus.cc:695] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:38.577535 16925 consensus_queue.cc:237] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.580471 16875 consensus_queue.cc:1035] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:38.588419 16829 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:38.588824 16829 ts_tablet_manager.cc:1397] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.160s user 0.009s sys 0.005s
I20250112 20:57:38.589346 16830 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:38.589653 16830 ts_tablet_manager.cc:1397] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.143s user 0.015s sys 0.017s
I20250112 20:57:38.591018 16861 consensus_queue.cc:1035] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:38.592428 16830 raft_consensus.cc:357] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.593106 16830 raft_consensus.cc:383] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.593380 16830 raft_consensus.cc:738] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.593957 16828 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:38.594276 16828 ts_tablet_manager.cc:1397] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.097s user 0.020s sys 0.005s
I20250112 20:57:38.595696 16830 consensus_queue.cc:260] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.596544 16861 raft_consensus.cc:491] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.597126 16861 raft_consensus.cc:513] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.599503 16861 leader_election.cc:290] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.602890 16830 ts_tablet_manager.cc:1428] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.013s user 0.006s sys 0.001s
I20250112 20:57:38.605561 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b544e7e4fa4e419949c833428c406b" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:38.605906 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b544e7e4fa4e419949c833428c406b" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:38.606165 16786 raft_consensus.cc:2463] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:38.606560 16636 raft_consensus.cc:2463] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:38.604420 16830 tablet_bootstrap.cc:492] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:38.608392 16680 leader_election.cc:304] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:38.609395 16861 raft_consensus.cc:2798] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:38.609836 16861 raft_consensus.cc:491] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:38.610248 16861 raft_consensus.cc:3054] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.622728 16894 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:38.623761 16892 consensus_queue.cc:1035] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:38.635115 16829 raft_consensus.cc:357] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.636722 16829 raft_consensus.cc:383] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.637063 16829 raft_consensus.cc:738] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.637626 16828 raft_consensus.cc:357] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.637730 16829 consensus_queue.cc:260] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.632898 16861 raft_consensus.cc:513] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.638316 16828 raft_consensus.cc:383] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.643913 16828 raft_consensus.cc:738] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.644532 16828 consensus_queue.cc:260] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.640480 16830 tablet_bootstrap.cc:654] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.653203 16829 ts_tablet_manager.cc:1428] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.064s user 0.005s sys 0.001s
I20250112 20:57:38.654035 16829 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:38.668922 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b544e7e4fa4e419949c833428c406b" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:38.669507 16638 raft_consensus.cc:3054] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.679069 16828 ts_tablet_manager.cc:1428] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.085s user 0.003s sys 0.003s
I20250112 20:57:38.675196 16891 raft_consensus.cc:491] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.680821 16828 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:38.680869 16861 leader_election.cc:290] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.683030 16891 raft_consensus.cc:513] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.697907 16638 raft_consensus.cc:2463] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:38.699025 16678 leader_election.cc:304] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:38.699872 16861 raft_consensus.cc:2798] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:38.700327 16861 raft_consensus.cc:695] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:38.700748 16925 consensus_queue.cc:1035] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:38.701071 16861 consensus_queue.cc:237] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.694864 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86b544e7e4fa4e419949c833428c406b" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:38.711503 16537 catalog_manager.cc:5526] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:38.715848 16787 raft_consensus.cc:3054] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.717157 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:38.718395 16602 leader_election.cc:343] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.720355 16786 raft_consensus.cc:3054] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.721505 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:38.722810 16605 leader_election.cc:343] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.723306 16605 leader_election.cc:304] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:38.726213 16863 raft_consensus.cc:2743] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:38.728351 16894 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:38.720434 16891 leader_election.cc:290] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.739677 16829 tablet_bootstrap.cc:654] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.750980 16875 raft_consensus.cc:491] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.751535 16875 raft_consensus.cc:513] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.754312 16537 catalog_manager.cc:5526] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:38.757195 16863 raft_consensus.cc:491] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.758517 16830 tablet_bootstrap.cc:492] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:38.758165 16863 raft_consensus.cc:513] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.759043 16830 ts_tablet_manager.cc:1397] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.155s user 0.017s sys 0.008s
I20250112 20:57:38.761657 16863 leader_election.cc:290] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.763954 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:38.764511 16711 raft_consensus.cc:2463] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:38.764747 16787 raft_consensus.cc:2463] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:38.765419 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:38.766065 16788 raft_consensus.cc:2463] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:38.770046 16605 leader_election.cc:304] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:38.778913 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:38.779639 16638 raft_consensus.cc:2463] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:38.780828 16752 leader_election.cc:304] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:38.781636 16925 raft_consensus.cc:2798] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:38.782050 16925 raft_consensus.cc:491] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:38.782447 16925 raft_consensus.cc:3054] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.784981 16875 leader_election.cc:290] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:38.786489 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:38.788287 16751 leader_election.cc:343] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.773257 16891 raft_consensus.cc:2798] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:38.803396 16891 raft_consensus.cc:491] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:38.817207 16891 raft_consensus.cc:3054] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.828011 16830 raft_consensus.cc:357] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.828568 16830 raft_consensus.cc:383] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.828809 16830 raft_consensus.cc:738] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.829203 16830 consensus_queue.cc:260] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:38.831396 16968 log.cc:926] Time spent T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20250112 20:57:38.839114 16925 raft_consensus.cc:513] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.849773 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:38.850361 16635 raft_consensus.cc:3054] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.852017 16830 ts_tablet_manager.cc:1428] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.093s user 0.004s sys 0.008s
I20250112 20:57:38.852234 16925 leader_election.cc:290] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:38.852839 16830 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:38.867820 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
W20250112 20:57:38.868882 16751 leader_election.cc:343] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.869751 16891 raft_consensus.cc:513] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.871199 16891 leader_election.cc:290] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.871780 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:38.872287 16711 raft_consensus.cc:3054] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.872176 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:38.872861 16786 raft_consensus.cc:3054] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.876524 16711 raft_consensus.cc:2463] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:38.877625 16602 leader_election.cc:304] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:38.878193 16891 raft_consensus.cc:2798] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:38.878633 16891 raft_consensus.cc:695] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:38.879278 16891 consensus_queue.cc:237] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.887494 16828 tablet_bootstrap.cc:654] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.889113 16786 raft_consensus.cc:2463] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:38.895722 16635 raft_consensus.cc:2463] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:38.896286 16891 raft_consensus.cc:491] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:38.896916 16752 leader_election.cc:304] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:38.896829 16891 raft_consensus.cc:513] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.897332 16536 catalog_manager.cc:5526] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:38.898993 16891 leader_election.cc:290] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:38.899952 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:38.901202 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:38.901315 16605 leader_election.cc:343] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:38.902516 16602 leader_election.cc:343] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:38.902943 16602 leader_election.cc:304] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:38.903693 16891 raft_consensus.cc:2743] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:38.904335 16875 raft_consensus.cc:2798] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:38.904853 16875 raft_consensus.cc:695] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:38.905697 16875 consensus_queue.cc:237] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.907282 16828 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:38.907730 16828 ts_tablet_manager.cc:1397] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.227s user 0.011s sys 0.014s
I20250112 20:57:38.889096 16829 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:38.909760 16829 ts_tablet_manager.cc:1397] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.256s user 0.009s sys 0.034s
I20250112 20:57:38.910171 16828 raft_consensus.cc:357] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.910844 16828 raft_consensus.cc:383] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.911103 16828 raft_consensus.cc:738] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.912001 16829 raft_consensus.cc:357] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.912917 16829 raft_consensus.cc:383] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.913182 16829 raft_consensus.cc:738] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.913866 16829 consensus_queue.cc:260] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.919701 16829 ts_tablet_manager.cc:1428] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20250112 20:57:38.920188 16828 consensus_queue.cc:260] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.921710 16829 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:38.924866 16830 tablet_bootstrap.cc:654] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.930718 16828 ts_tablet_manager.cc:1428] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20250112 20:57:38.931649 16828 tablet_bootstrap.cc:492] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:38.937288 16786 raft_consensus.cc:3054] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:38.966600 16828 tablet_bootstrap.cc:654] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.967689 16535 catalog_manager.cc:5526] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:38.971531 16829 tablet_bootstrap.cc:654] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:38.979203 16830 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:38.979641 16830 ts_tablet_manager.cc:1397] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.127s user 0.009s sys 0.003s
I20250112 20:57:38.982142 16830 raft_consensus.cc:357] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:38.982823 16830 raft_consensus.cc:383] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:38.983079 16830 raft_consensus.cc:738] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:38.983799 16830 consensus_queue.cc:260] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.000994 16828 tablet_bootstrap.cc:492] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.001457 16828 ts_tablet_manager.cc:1397] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.070s user 0.012s sys 0.000s
I20250112 20:57:39.006577 16892 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:39.014669 16828 raft_consensus.cc:357] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.015388 16828 raft_consensus.cc:383] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.015651 16828 raft_consensus.cc:738] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.016268 16828 consensus_queue.cc:260] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.028357 16830 ts_tablet_manager.cc:1428] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.048s user 0.005s sys 0.001s
I20250112 20:57:39.029312 16830 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:39.035265 16829 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:39.035883 16829 ts_tablet_manager.cc:1397] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.115s user 0.006s sys 0.004s
I20250112 20:57:39.038219 16829 raft_consensus.cc:357] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.038856 16829 raft_consensus.cc:383] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.039098 16829 raft_consensus.cc:738] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.039757 16829 consensus_queue.cc:260] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.040815 16875 raft_consensus.cc:491] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.041221 16875 raft_consensus.cc:513] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.043170 16875 leader_election.cc:290] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.043320 16828 ts_tablet_manager.cc:1428] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20250112 20:57:39.045153 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.045809 16711 raft_consensus.cc:2463] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:39.045166 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:39.047112 16638 raft_consensus.cc:2463] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:39.048336 16752 leader_election.cc:304] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:39.049815 16875 raft_consensus.cc:2798] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:39.050177 16875 raft_consensus.cc:491] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:39.050275 16638 raft_consensus.cc:3054] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.050563 16875 raft_consensus.cc:3054] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.057159 16875 raft_consensus.cc:513] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.064584 16875 leader_election.cc:290] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.064875 16925 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:39.066792 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:39.067430 16636 raft_consensus.cc:3054] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.075503 16828 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.076691 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:39.077315 16711 raft_consensus.cc:3054] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.089351 16828 tablet_bootstrap.cc:654] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:39.098006 16680 consensus_peers.cc:487] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:39.111038 16828 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.111480 16828 ts_tablet_manager.cc:1397] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.019s
I20250112 20:57:39.111910 16853 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:39.113997 16828 raft_consensus.cc:357] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.114675 16828 raft_consensus.cc:383] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.114921 16828 raft_consensus.cc:738] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.115489 16828 consensus_queue.cc:260] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.125936 16828 ts_tablet_manager.cc:1428] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20250112 20:57:39.126993 16828 tablet_bootstrap.cc:492] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.129432 16636 raft_consensus.cc:2463] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:39.130590 16752 leader_election.cc:304] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:39.131491 16853 raft_consensus.cc:2798] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:39.131954 16853 raft_consensus.cc:695] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:39.132803 16853 consensus_queue.cc:237] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.133714 16828 tablet_bootstrap.cc:654] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.139906 16829 ts_tablet_manager.cc:1428] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.104s user 0.005s sys 0.001s
I20250112 20:57:39.147363 16711 raft_consensus.cc:2463] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
W20250112 20:57:39.148625 16980 log.cc:926] Time spent T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: Append to log took a long time: real 0.107s user 0.000s sys 0.001s
I20250112 20:57:39.152014 16828 tablet_bootstrap.cc:492] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.152508 16828 ts_tablet_manager.cc:1397] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.026s user 0.014s sys 0.003s
I20250112 20:57:39.155148 16829 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:39.156307 16828 raft_consensus.cc:357] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.157130 16828 raft_consensus.cc:383] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.157421 16828 raft_consensus.cc:738] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.158233 16828 consensus_queue.cc:260] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.160704 16828 ts_tablet_manager.cc:1428] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20250112 20:57:39.161623 16828 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.169160 16830 tablet_bootstrap.cc:654] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.171736 16828 tablet_bootstrap.cc:654] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.181877 16894 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:39.187961 16536 catalog_manager.cc:5526] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:39.222440 16892 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:39.228111 16829 tablet_bootstrap.cc:654] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.245680 16830 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:39.246099 16830 ts_tablet_manager.cc:1397] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.217s user 0.011s sys 0.005s
I20250112 20:57:39.247965 16828 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.248423 16828 ts_tablet_manager.cc:1397] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.087s user 0.019s sys 0.011s
I20250112 20:57:39.249971 16830 raft_consensus.cc:357] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.250684 16830 raft_consensus.cc:383] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.250933 16830 raft_consensus.cc:738] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.251509 16830 consensus_queue.cc:260] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.253506 16829 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:39.253893 16829 ts_tablet_manager.cc:1397] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.113s user 0.013s sys 0.001s
I20250112 20:57:39.255492 16830 ts_tablet_manager.cc:1428] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20250112 20:57:39.258641 16829 raft_consensus.cc:357] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.259111 16829 raft_consensus.cc:383] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.259311 16829 raft_consensus.cc:738] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.259814 16829 consensus_queue.cc:260] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.261879 16828 raft_consensus.cc:357] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.262530 16828 raft_consensus.cc:383] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.262775 16828 raft_consensus.cc:738] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.267647 16830 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:39.272847 16828 consensus_queue.cc:260] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.279675 16830 tablet_bootstrap.cc:654] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.280769 16891 raft_consensus.cc:491] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.281194 16891 raft_consensus.cc:513] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.282693 16891 leader_election.cc:290] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.283394 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132239c0c07a4c2f9334e84188f329b3" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.284022 16714 raft_consensus.cc:2463] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:39.285419 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132239c0c07a4c2f9334e84188f329b3" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:39.286237 16602 leader_election.cc:304] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:39.285102 16947 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:39.288089 16605 leader_election.cc:343] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.288805 16947 raft_consensus.cc:2798] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:39.289155 16947 raft_consensus.cc:491] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:39.289454 16947 raft_consensus.cc:3054] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.291754 16829 ts_tablet_manager.cc:1428] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.038s user 0.006s sys 0.000s
I20250112 20:57:39.292467 16829 tablet_bootstrap.cc:492] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
W20250112 20:57:39.295506 16751 consensus_peers.cc:487] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:39.303867 16947 raft_consensus.cc:513] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.316679 16829 tablet_bootstrap.cc:654] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.318356 16828 ts_tablet_manager.cc:1428] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.070s user 0.005s sys 0.000s
I20250112 20:57:39.319284 16828 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.319255 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132239c0c07a4c2f9334e84188f329b3" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:39.325158 16714 raft_consensus.cc:3054] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.333250 16830 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:39.333714 16830 ts_tablet_manager.cc:1397] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.066s user 0.014s sys 0.013s
I20250112 20:57:39.337693 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132239c0c07a4c2f9334e84188f329b3" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:39.345232 16830 raft_consensus.cc:357] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.345929 16830 raft_consensus.cc:383] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.346186 16830 raft_consensus.cc:738] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.346822 16830 consensus_queue.cc:260] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.333674 16947 leader_election.cc:290] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.348795 16891 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250112 20:57:39.355170 16830 ts_tablet_manager.cc:1428] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20250112 20:57:39.356257 16830 tablet_bootstrap.cc:492] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:39.359505 16714 raft_consensus.cc:2463] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
W20250112 20:57:39.350206 16605 leader_election.cc:343] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.365973 16602 leader_election.cc:304] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:39.371069 16828 tablet_bootstrap.cc:654] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.371464 16863 raft_consensus.cc:2798] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:39.376727 16863 raft_consensus.cc:695] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:39.379614 16863 consensus_queue.cc:237] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.385020 16853 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:39.387619 16829 tablet_bootstrap.cc:492] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:39.388044 16829 ts_tablet_manager.cc:1397] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.096s user 0.029s sys 0.012s
I20250112 20:57:39.401690 16828 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.404279 16967 raft_consensus.cc:491] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.404788 16967 raft_consensus.cc:513] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.406844 16967 leader_election.cc:290] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.403856 16828 ts_tablet_manager.cc:1397] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.085s user 0.015s sys 0.000s
I20250112 20:57:39.403821 16829 raft_consensus.cc:357] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.408190 16829 raft_consensus.cc:383] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.408526 16829 raft_consensus.cc:738] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.409837 16829 consensus_queue.cc:260] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.413136 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:39.413399 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
W20250112 20:57:39.414362 16680 leader_election.cc:343] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:39.416677 16678 leader_election.cc:343] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.417086 16678 leader_election.cc:304] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:39.419651 16967 raft_consensus.cc:2743] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:39.419739 16829 ts_tablet_manager.cc:1428] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.031s user 0.004s sys 0.004s
I20250112 20:57:39.421625 16829 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:39.423537 16830 tablet_bootstrap.cc:654] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.424276 16536 catalog_manager.cc:5526] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:39.430716 16967 raft_consensus.cc:491] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.431787 16967 raft_consensus.cc:513] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.432952 16829 tablet_bootstrap.cc:654] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:39.437052 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:39.437855 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:39.434543 16967 leader_election.cc:290] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:39.439889 16680 leader_election.cc:343] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.444764 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:39.444804 16828 raft_consensus.cc:357] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.448732 16828 raft_consensus.cc:383] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.449035 16828 raft_consensus.cc:738] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
W20250112 20:57:39.449156 16678 leader_election.cc:343] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.449618 16678 leader_election.cc:304] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:39.450338 16967 raft_consensus.cc:2743] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:39.449898 16828 consensus_queue.cc:260] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.453902 16828 ts_tablet_manager.cc:1428] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.046s user 0.007s sys 0.000s
I20250112 20:57:39.455399 16828 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.467854 16829 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:39.468397 16829 ts_tablet_manager.cc:1397] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.047s user 0.013s sys 0.009s
I20250112 20:57:39.468469 16875 raft_consensus.cc:491] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.469785 16828 tablet_bootstrap.cc:654] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.469545 16875 raft_consensus.cc:513] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.471797 16875 leader_election.cc:290] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.472225 16830 tablet_bootstrap.cc:492] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:39.471868 16829 raft_consensus.cc:357] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.472470 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:39.472625 16829 raft_consensus.cc:383] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.472652 16830 ts_tablet_manager.cc:1397] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.117s user 0.011s sys 0.014s
I20250112 20:57:39.473008 16829 raft_consensus.cc:738] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.473733 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.473604 16829 consensus_queue.cc:260] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.475818 16751 leader_election.cc:304] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:39.477654 16925 raft_consensus.cc:2743] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:39.479487 16830 raft_consensus.cc:357] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.479946 16829 ts_tablet_manager.cc:1428] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.011s user 0.003s sys 0.003s
I20250112 20:57:39.480203 16830 raft_consensus.cc:383] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.480518 16830 raft_consensus.cc:738] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.480846 16829 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:39.483485 16828 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.484201 16828 ts_tablet_manager.cc:1397] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.000s
I20250112 20:57:39.484836 16830 consensus_queue.cc:260] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.486737 16828 raft_consensus.cc:357] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.487422 16828 raft_consensus.cc:383] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.487694 16828 raft_consensus.cc:738] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.488262 16828 consensus_queue.cc:260] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.490520 16828 ts_tablet_manager.cc:1428] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250112 20:57:39.492307 16830 ts_tablet_manager.cc:1428] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20250112 20:57:39.493109 16830 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:39.493943 16829 tablet_bootstrap.cc:654] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.500222 16828 tablet_bootstrap.cc:492] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.502669 16830 tablet_bootstrap.cc:654] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.508026 16828 tablet_bootstrap.cc:654] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.523048 16830 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:39.523501 16830 ts_tablet_manager.cc:1397] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.000s
I20250112 20:57:39.525914 16863 raft_consensus.cc:491] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.526037 16829 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:39.526641 16829 ts_tablet_manager.cc:1397] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.046s user 0.012s sys 0.008s
I20250112 20:57:39.526391 16830 raft_consensus.cc:357] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.526422 16863 raft_consensus.cc:513] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.527256 16830 raft_consensus.cc:383] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.527587 16830 raft_consensus.cc:738] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.527690 16828 tablet_bootstrap.cc:492] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.528127 16828 ts_tablet_manager.cc:1397] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.028s user 0.015s sys 0.002s
I20250112 20:57:39.528180 16830 consensus_queue.cc:260] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.530663 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d979f5162b594f329ba9439c121741de" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.530942 16828 raft_consensus.cc:357] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.531342 16714 raft_consensus.cc:2463] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:39.531575 16828 raft_consensus.cc:383] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.531860 16828 raft_consensus.cc:738] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.532407 16828 consensus_queue.cc:260] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.532573 16602 leader_election.cc:304] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:39.533646 16829 raft_consensus.cc:357] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.534412 16829 raft_consensus.cc:383] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.534550 16830 ts_tablet_manager.cc:1428] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20250112 20:57:39.534498 16828 ts_tablet_manager.cc:1428] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20250112 20:57:39.534793 16853 raft_consensus.cc:491] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.534746 16829 raft_consensus.cc:738] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.535490 16830 tablet_bootstrap.cc:492] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:39.535583 16828 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.535665 16829 consensus_queue.cc:260] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.537923 16829 ts_tablet_manager.cc:1428] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250112 20:57:39.539979 16829 tablet_bootstrap.cc:492] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:39.540710 16861 raft_consensus.cc:491] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.541154 16861 raft_consensus.cc:513] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.543062 16861 leader_election.cc:290] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.533874 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d979f5162b594f329ba9439c121741de" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:39.535331 16853 raft_consensus.cc:513] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.528537 16863 leader_election.cc:290] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.548115 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:39.551276 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:39.554421 16853 leader_election.cc:290] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.557158 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.558012 16714 raft_consensus.cc:2463] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:39.533900 16891 raft_consensus.cc:2798] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:39.559041 16891 raft_consensus.cc:491] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:39.559955 16891 raft_consensus.cc:3054] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.565034 16787 raft_consensus.cc:2463] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:39.566416 16863 raft_consensus.cc:491] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.566629 16678 leader_election.cc:304] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:39.567600 16861 raft_consensus.cc:491] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.573226 16830 tablet_bootstrap.cc:654] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.566967 16863 raft_consensus.cc:513] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.576901 16947 raft_consensus.cc:491] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.577607 16828 tablet_bootstrap.cc:654] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.577322 16947 raft_consensus.cc:513] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.579676 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:39.580327 16637 raft_consensus.cc:2463] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:39.580722 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:39.581013 16751 leader_election.cc:304] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:39.585508 16925 raft_consensus.cc:2798] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:39.585860 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19f4890a44f4f088549ad693bafbf8d" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.585884 16925 raft_consensus.cc:491] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:39.586930 16925 raft_consensus.cc:3054] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.581735 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.587569 16714 raft_consensus.cc:2463] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:39.589421 16891 raft_consensus.cc:513] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.589850 16602 leader_election.cc:304] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:39.576285 16892 raft_consensus.cc:2743] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:39.590759 16993 raft_consensus.cc:2798] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:39.591106 16993 raft_consensus.cc:491] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:39.591436 16993 raft_consensus.cc:3054] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.568164 16861 raft_consensus.cc:513] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.591921 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d979f5162b594f329ba9439c121741de" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:39.592489 16714 raft_consensus.cc:3054] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.593497 16861 leader_election.cc:290] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.594430 16711 raft_consensus.cc:2463] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:39.595255 16891 leader_election.cc:290] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.580348 16947 leader_election.cc:290] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.597921 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d979f5162b594f329ba9439c121741de" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:39.598496 16788 raft_consensus.cc:3054] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.598990 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19f4890a44f4f088549ad693bafbf8d" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:39.599561 16635 raft_consensus.cc:2463] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:39.600251 16602 leader_election.cc:304] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:39.601680 16947 raft_consensus.cc:2798] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:39.602046 16947 raft_consensus.cc:491] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20250112 20:57:39.602466 16605 leader_election.cc:343] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.602797 16947 raft_consensus.cc:3054] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.595639 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19f4890a44f4f088549ad693bafbf8d" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:39.604770 16680 leader_election.cc:343] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.606683 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19f4890a44f4f088549ad693bafbf8d" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:39.582176 16863 leader_election.cc:290] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.607836 16993 raft_consensus.cc:513] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:39.614104 16605 leader_election.cc:343] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.615274 16993 leader_election.cc:290] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.615913 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:39.616452 16711 raft_consensus.cc:3054] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.618880 16947 raft_consensus.cc:513] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.619017 16925 raft_consensus.cc:513] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.620688 16947 leader_election.cc:290] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.621119 16678 leader_election.cc:304] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:39.621471 16710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19f4890a44f4f088549ad693bafbf8d" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:39.622099 16710 raft_consensus.cc:3054] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.623107 16829 tablet_bootstrap.cc:654] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.626667 16788 raft_consensus.cc:2463] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:39.629599 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e19f4890a44f4f088549ad693bafbf8d" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:39.630054 16605 leader_election.cc:304] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:39.629585 16853 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250112 20:57:39.630937 16605 leader_election.cc:343] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.631803 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:39.631690 16710 raft_consensus.cc:2463] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:39.632418 16892 raft_consensus.cc:2758] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20250112 20:57:39.612983 16714 raft_consensus.cc:2463] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:39.635645 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
W20250112 20:57:39.635932 16605 leader_election.cc:343] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.636221 16637 raft_consensus.cc:3054] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.636474 16925 leader_election.cc:290] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.637214 16602 leader_election.cc:304] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:39.638211 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:39.655193 16853 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20250112 20:57:39.688925 16713 raft_consensus.cc:3054] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.638293 16711 raft_consensus.cc:2463] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:39.687593 16830 tablet_bootstrap.cc:492] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:39.691344 16602 leader_election.cc:304] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:39.691716 16830 ts_tablet_manager.cc:1397] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.156s user 0.019s sys 0.014s
I20250112 20:57:39.646859 16637 raft_consensus.cc:2463] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:39.638111 16993 raft_consensus.cc:2798] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:39.643294 16947 raft_consensus.cc:2798] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:39.694790 16891 raft_consensus.cc:2798] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:39.693885 16993 raft_consensus.cc:695] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:39.701296 16752 leader_election.cc:304] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:39.703579 16925 raft_consensus.cc:2798] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:39.704011 16925 raft_consensus.cc:695] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:39.710427 16993 consensus_queue.cc:237] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.713120 16925 consensus_queue.cc:237] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.719105 16830 raft_consensus.cc:357] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.719718 16787 raft_consensus.cc:3054] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.725375 16830 raft_consensus.cc:383] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.726256 16830 raft_consensus.cc:738] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.730852 16830 consensus_queue.cc:260] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.737358 16829 tablet_bootstrap.cc:492] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:39.737936 16829 ts_tablet_manager.cc:1397] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.198s user 0.003s sys 0.020s
I20250112 20:57:39.740293 16830 ts_tablet_manager.cc:1428] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.048s user 0.003s sys 0.003s
I20250112 20:57:39.741521 16830 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:39.734735 16891 raft_consensus.cc:695] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:39.738312 16947 raft_consensus.cc:695] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:39.750875 16829 raft_consensus.cc:357] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.751847 16829 raft_consensus.cc:383] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.752120 16829 raft_consensus.cc:738] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.753903 16829 consensus_queue.cc:260] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.755064 16828 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.755439 16828 ts_tablet_manager.cc:1397] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.220s user 0.015s sys 0.010s
I20250112 20:57:39.754797 16536 catalog_manager.cc:5526] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:39.759699 16710 raft_consensus.cc:3054] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.772769 16829 ts_tablet_manager.cc:1428] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20250112 20:57:39.774513 16829 tablet_bootstrap.cc:492] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:39.773137 16947 consensus_queue.cc:237] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.776221 16828 raft_consensus.cc:357] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.778905 16828 raft_consensus.cc:383] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.779249 16828 raft_consensus.cc:738] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.780622 16828 consensus_queue.cc:260] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.811703 16829 tablet_bootstrap.cc:654] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.819038 16830 tablet_bootstrap.cc:654] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.785409 16891 consensus_queue.cc:237] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.821921 16535 catalog_manager.cc:5526] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:39.823736 16828 ts_tablet_manager.cc:1428] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.068s user 0.009s sys 0.002s
I20250112 20:57:39.824672 16828 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.833824 16875 raft_consensus.cc:491] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.834266 16875 raft_consensus.cc:513] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.836822 16875 leader_election.cc:290] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.838761 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:39.840200 16710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.840889 16710 raft_consensus.cc:2463] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:39.841974 16751 leader_election.cc:304] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:39.842809 16875 raft_consensus.cc:2798] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:39.843174 16875 raft_consensus.cc:491] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:39.843555 16875 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:57:39.849848 16752 leader_election.cc:343] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.851600 16713 raft_consensus.cc:2463] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:39.853478 16925 raft_consensus.cc:491] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.854292 16925 raft_consensus.cc:513] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.854703 16875 raft_consensus.cc:513] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.856223 16925 leader_election.cc:290] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.856858 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3b5e20d22c4251b1e2b91ca0479e2f" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:39.857215 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3b5e20d22c4251b1e2b91ca0479e2f" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:39.857869 16713 raft_consensus.cc:2463] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
W20250112 20:57:39.858225 16752 leader_election.cc:343] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.858806 16751 leader_election.cc:304] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:39.860278 16925 raft_consensus.cc:2798] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:39.860749 16925 raft_consensus.cc:491] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:39.861065 16925 raft_consensus.cc:3054] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.861979 16830 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:39.862414 16830 ts_tablet_manager.cc:1397] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.121s user 0.008s sys 0.011s
I20250112 20:57:39.860219 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:39.865597 16830 raft_consensus.cc:357] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.866266 16830 raft_consensus.cc:383] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.866536 16830 raft_consensus.cc:738] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.867126 16830 consensus_queue.cc:260] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.868490 16829 tablet_bootstrap.cc:492] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:39.868932 16829 ts_tablet_manager.cc:1397] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.095s user 0.007s sys 0.011s
W20250112 20:57:39.869621 16752 leader_election.cc:343] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.871168 16829 raft_consensus.cc:357] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.871821 16829 raft_consensus.cc:383] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.872125 16829 raft_consensus.cc:738] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.874119 16925 raft_consensus.cc:513] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.875193 16828 tablet_bootstrap.cc:654] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.876628 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3b5e20d22c4251b1e2b91ca0479e2f" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:39.876834 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3b5e20d22c4251b1e2b91ca0479e2f" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:39.881572 16713 raft_consensus.cc:3054] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:57:39.881973 16752 leader_election.cc:343] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.883008 16875 leader_election.cc:290] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.884006 16710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:39.884691 16710 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:39.877266 16925 leader_election.cc:290] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:39.872726 16829 consensus_queue.cc:260] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.878551 16830 ts_tablet_manager.cc:1428] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.016s user 0.006s sys 0.002s
I20250112 20:57:39.891033 16830 tablet_bootstrap.cc:492] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:39.891130 16828 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.891628 16828 ts_tablet_manager.cc:1397] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.067s user 0.005s sys 0.005s
I20250112 20:57:39.893287 16713 raft_consensus.cc:2463] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:39.894081 16828 raft_consensus.cc:357] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.894703 16829 ts_tablet_manager.cc:1428] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20250112 20:57:39.894841 16828 raft_consensus.cc:383] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.895172 16828 raft_consensus.cc:738] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.895696 16751 leader_election.cc:304] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:39.895797 16828 consensus_queue.cc:260] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.897686 16710 raft_consensus.cc:2463] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:39.898550 16537 catalog_manager.cc:5526] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:39.903710 16751 leader_election.cc:304] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:39.904830 17005 raft_consensus.cc:2798] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:39.905817 17005 raft_consensus.cc:695] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:39.906378 16828 ts_tablet_manager.cc:1428] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20250112 20:57:39.906553 17005 consensus_queue.cc:237] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.902164 16875 raft_consensus.cc:2798] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:39.909866 16875 raft_consensus.cc:695] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:39.910636 16875 consensus_queue.cc:237] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.909516 16828 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:39.921584 16537 catalog_manager.cc:5526] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:39.923789 16828 tablet_bootstrap.cc:654] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.926091 16829 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
W20250112 20:57:39.926637 16605 consensus_peers.cc:487] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:39.933800 16830 tablet_bootstrap.cc:654] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.941969 16829 tablet_bootstrap.cc:654] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:39.948985 16536 catalog_manager.cc:5526] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:39.951733 16993 consensus_queue.cc:1035] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:39.951143 16536 catalog_manager.cc:5526] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:39.967855 16993 raft_consensus.cc:491] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:39.968371 16993 raft_consensus.cc:513] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.976975 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:39.978368 16605 leader_election.cc:343] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.979281 16993 leader_election.cc:290] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:39.979434 16828 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:39.979910 16828 ts_tablet_manager.cc:1397] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.071s user 0.009s sys 0.011s
I20250112 20:57:39.979919 16710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:39.981371 16602 leader_election.cc:343] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:39.981909 16829 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:39.981755 16602 leader_election.cc:304] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:39.982286 16829 ts_tablet_manager.cc:1397] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.056s user 0.011s sys 0.006s
I20250112 20:57:39.983023 16993 raft_consensus.cc:2743] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:39.984377 16829 raft_consensus.cc:357] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.984984 16829 raft_consensus.cc:383] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.986657 16830 tablet_bootstrap.cc:492] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:39.986997 16830 ts_tablet_manager.cc:1397] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.096s user 0.009s sys 0.009s
I20250112 20:57:39.989006 16830 raft_consensus.cc:357] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.989634 16830 raft_consensus.cc:383] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:39.989877 16830 raft_consensus.cc:738] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.990458 16828 raft_consensus.cc:357] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.985278 16829 raft_consensus.cc:738] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:39.991022 16830 consensus_queue.cc:260] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.991766 16829 consensus_queue.cc:260] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:39.997138 16830 ts_tablet_manager.cc:1428] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20250112 20:57:39.991932 16828 raft_consensus.cc:383] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.001931 16828 raft_consensus.cc:738] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.002575 16828 consensus_queue.cc:260] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.001547 16829 ts_tablet_manager.cc:1428] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.019s user 0.005s sys 0.002s
I20250112 20:57:40.004177 16830 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:40.021726 16828 ts_tablet_manager.cc:1428] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.042s user 0.004s sys 0.004s
I20250112 20:57:40.022871 16828 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:40.023082 16829 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:40.032375 16828 tablet_bootstrap.cc:654] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.038363 16830 tablet_bootstrap.cc:654] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.054081 16829 tablet_bootstrap.cc:654] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.056671 16830 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:40.056787 16828 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:40.057113 16830 ts_tablet_manager.cc:1397] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.053s user 0.016s sys 0.000s
I20250112 20:57:40.057430 16828 ts_tablet_manager.cc:1397] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.009s
I20250112 20:57:40.059514 16830 raft_consensus.cc:357] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.060230 16830 raft_consensus.cc:383] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.060513 16830 raft_consensus.cc:738] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.061204 16830 consensus_queue.cc:260] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.059777 16828 raft_consensus.cc:357] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.064882 16828 raft_consensus.cc:383] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.065196 16828 raft_consensus.cc:738] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.065899 16828 consensus_queue.cc:260] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.068161 16828 ts_tablet_manager.cc:1428] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.010s user 0.002s sys 0.003s
I20250112 20:57:40.069121 16828 tablet_bootstrap.cc:492] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:40.075865 16828 tablet_bootstrap.cc:654] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.081732 16830 ts_tablet_manager.cc:1428] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20250112 20:57:40.082911 16830 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:40.086941 16891 raft_consensus.cc:491] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.087482 16891 raft_consensus.cc:513] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.089241 16830 tablet_bootstrap.cc:654] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.089470 16891 leader_election.cc:290] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.091042 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ffa8e639d640a3b7f19d9857148f40" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:40.091701 16713 raft_consensus.cc:2463] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:40.093371 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ffa8e639d640a3b7f19d9857148f40" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:40.093748 16602 leader_election.cc:304] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:40.094602 16891 raft_consensus.cc:2798] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
W20250112 20:57:40.094822 16605 leader_election.cc:343] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.094974 16891 raft_consensus.cc:491] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:40.097314 16891 raft_consensus.cc:3054] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.106076 16829 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:40.106956 16829 ts_tablet_manager.cc:1397] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.084s user 0.016s sys 0.004s
I20250112 20:57:40.109648 16829 raft_consensus.cc:357] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.110344 16829 raft_consensus.cc:383] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.110600 16829 raft_consensus.cc:738] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.111759 16829 consensus_queue.cc:260] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.113969 16829 ts_tablet_manager.cc:1428] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20250112 20:57:40.115016 16891 raft_consensus.cc:513] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.117744 16829 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:40.120333 16828 tablet_bootstrap.cc:492] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:40.120770 16828 ts_tablet_manager.cc:1397] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.052s user 0.012s sys 0.002s
I20250112 20:57:40.123837 16828 raft_consensus.cc:357] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.124439 16828 raft_consensus.cc:383] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.124748 16828 raft_consensus.cc:738] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.124855 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ffa8e639d640a3b7f19d9857148f40" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:40.125439 16713 raft_consensus.cc:3054] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.125372 16828 consensus_queue.cc:260] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.127552 16828 ts_tablet_manager.cc:1428] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20250112 20:57:40.129678 16830 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:40.130077 16830 ts_tablet_manager.cc:1397] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.047s user 0.010s sys 0.008s
I20250112 20:57:40.131415 17005 raft_consensus.cc:491] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.131852 17005 raft_consensus.cc:513] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.133666 17005 leader_election.cc:290] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:40.135295 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:40.140461 16713 raft_consensus.cc:2463] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:40.141522 16602 leader_election.cc:304] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:40.142357 16947 raft_consensus.cc:2798] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:40.132396 16830 raft_consensus.cc:357] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.142745 16830 raft_consensus.cc:383] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.142794 16947 raft_consensus.cc:695] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:40.143072 16830 raft_consensus.cc:738] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.143677 16947 consensus_queue.cc:237] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.143720 16830 consensus_queue.cc:260] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.146456 16830 ts_tablet_manager.cc:1428] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20250112 20:57:40.147332 16830 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:40.148806 16710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:40.150455 16710 raft_consensus.cc:2388] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c29b243acf2b4f248606563fc96efc7d in current term 1: Already voted for candidate 80770068c3a34f918765f54dbe3a94eb in this term.
I20250112 20:57:40.151557 16751 leader_election.cc:304] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:40.152828 16875 raft_consensus.cc:3054] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.154939 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ffa8e639d640a3b7f19d9857148f40" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:40.155951 16894 raft_consensus.cc:491] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250112 20:57:40.156212 16605 leader_election.cc:343] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.156474 16894 raft_consensus.cc:513] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.157207 16891 leader_election.cc:290] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.159054 16894 leader_election.cc:290] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.164186 16828 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:40.165310 17005 raft_consensus.cc:491] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.165546 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:40.165783 17005 raft_consensus.cc:513] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.166147 16635 raft_consensus.cc:2463] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:40.167358 16678 leader_election.cc:304] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:40.165177 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:40.168221 16875 raft_consensus.cc:2743] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:40.171808 16894 raft_consensus.cc:2798] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:40.172444 16894 raft_consensus.cc:491] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20250112 20:57:40.172394 16680 leader_election.cc:343] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.173067 16894 raft_consensus.cc:3054] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.175285 17005 leader_election.cc:290] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:40.183213 16829 tablet_bootstrap.cc:654] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.184233 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:40.186667 16635 raft_consensus.cc:2463] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:40.190941 16752 leader_election.cc:304] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:40.191771 17005 raft_consensus.cc:2798] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:40.192122 17005 raft_consensus.cc:491] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:40.192445 17005 raft_consensus.cc:3054] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.188642 16535 catalog_manager.cc:5526] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:40.195307 16830 tablet_bootstrap.cc:654] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.190634 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:40.197345 16967 raft_consensus.cc:491] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.200685 16967 raft_consensus.cc:513] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.200984 16894 raft_consensus.cc:513] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.202793 16894 leader_election.cc:290] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.203346 16967 leader_election.cc:290] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:40.204479 16751 leader_election.cc:343] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.208849 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c34b0c62d424c9e9f764d74f6baf1ba" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:40.209441 16637 raft_consensus.cc:2463] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:40.210300 16828 tablet_bootstrap.cc:654] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.213625 16829 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:40.214039 16829 ts_tablet_manager.cc:1397] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.096s user 0.009s sys 0.005s
I20250112 20:57:40.212016 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:40.215607 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c34b0c62d424c9e9f764d74f6baf1ba" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:40.216516 16678 leader_election.cc:304] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
W20250112 20:57:40.216836 16680 leader_election.cc:343] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.217478 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:40.217438 16829 raft_consensus.cc:357] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.217314 16894 raft_consensus.cc:2798] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:40.218211 16829 raft_consensus.cc:383] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20250112 20:57:40.217490 16680 leader_election.cc:343] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.219713 16829 raft_consensus.cc:738] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.220775 16829 consensus_queue.cc:260] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.218231 16635 raft_consensus.cc:3054] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.222834 16829 ts_tablet_manager.cc:1428] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20250112 20:57:40.223826 16829 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:40.220050 16894 raft_consensus.cc:491] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:40.225793 17005 raft_consensus.cc:513] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.227924 16894 raft_consensus.cc:3054] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.233297 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:40.235586 16638 raft_consensus.cc:3054] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.240662 16900 raft_consensus.cc:491] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.240939 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:40.241159 16900 raft_consensus.cc:513] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:40.242129 16751 leader_election.cc:343] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.244794 16861 raft_consensus.cc:491] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.245283 16861 raft_consensus.cc:513] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.246088 16900 leader_election.cc:290] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:40.246991 16829 tablet_bootstrap.cc:654] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.247285 16861 leader_election.cc:290] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.243712 17005 leader_election.cc:290] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:40.247924 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132239c0c07a4c2f9334e84188f329b3" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:40.248131 16638 raft_consensus.cc:2463] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:40.249943 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a13e4117fd64b15a4cd60943e638999" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:40.250582 16788 raft_consensus.cc:2463] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:40.251113 16752 leader_election.cc:304] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:40.251364 16635 raft_consensus.cc:2463] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:40.251943 17005 raft_consensus.cc:2798] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:40.252398 16680 leader_election.cc:304] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:40.252671 17005 raft_consensus.cc:695] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:40.254135 17005 consensus_queue.cc:237] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.254966 16861 raft_consensus.cc:2798] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:40.254586 16894 raft_consensus.cc:513] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.254462 16713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132239c0c07a4c2f9334e84188f329b3" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:40.255342 16861 raft_consensus.cc:491] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:40.258201 16861 raft_consensus.cc:3054] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.258544 16678 leader_election.cc:304] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:40.263001 16751 leader_election.cc:304] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:40.263801 16900 raft_consensus.cc:2743] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:40.264154 16967 raft_consensus.cc:2798] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:40.264674 16967 raft_consensus.cc:695] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:40.266177 16967 consensus_queue.cc:237] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.269701 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a13e4117fd64b15a4cd60943e638999" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
W20250112 20:57:40.270900 16678 leader_election.cc:343] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.273247 16828 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:40.273700 16828 ts_tablet_manager.cc:1397] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.112s user 0.012s sys 0.003s
I20250112 20:57:40.277860 16828 raft_consensus.cc:357] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.278519 16828 raft_consensus.cc:383] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.278782 16828 raft_consensus.cc:738] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.279325 16828 consensus_queue.cc:260] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.281657 16828 ts_tablet_manager.cc:1428] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250112 20:57:40.282594 16830 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:40.283062 16830 ts_tablet_manager.cc:1397] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.136s user 0.022s sys 0.000s
I20250112 20:57:40.282773 16900 consensus_queue.cc:1035] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:40.285629 16605 consensus_peers.cc:487] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:40.285497 16830 raft_consensus.cc:357] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.286203 16830 raft_consensus.cc:383] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.285629 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c34b0c62d424c9e9f764d74f6baf1ba" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:40.286782 16830 raft_consensus.cc:738] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.287564 16830 consensus_queue.cc:260] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:40.288648 16680 leader_election.cc:343] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.290457 16830 ts_tablet_manager.cc:1428] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20250112 20:57:40.291359 16830 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:40.291402 16891 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:40.282582 16828 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:40.285741 16894 leader_election.cc:290] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.301393 16638 raft_consensus.cc:3054] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.307579 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c34b0c62d424c9e9f764d74f6baf1ba" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:40.316540 16635 raft_consensus.cc:3054] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.314204 16861 raft_consensus.cc:513] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.318939 16534 catalog_manager.cc:5526] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:40.318761 16535 catalog_manager.cc:5526] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:40.326397 16861 leader_election.cc:290] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.323657 16993 consensus_queue.cc:1035] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250112 20:57:40.328991 16830 tablet_bootstrap.cc:654] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.329041 16993 consensus_queue.cc:1035] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:40.342607 16829 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:40.343211 16829 ts_tablet_manager.cc:1397] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.119s user 0.005s sys 0.027s
I20250112 20:57:40.347855 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a13e4117fd64b15a4cd60943e638999" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:40.346432 16925 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250112 20:57:40.348475 16786 raft_consensus.cc:3054] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.364601 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a13e4117fd64b15a4cd60943e638999" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:40.366519 16875 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250112 20:57:40.383502 16786 raft_consensus.cc:2463] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
W20250112 20:57:40.385160 16678 leader_election.cc:343] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:40.415110 17041 log.cc:926] Time spent T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: Append to log took a long time: real 0.084s user 0.000s sys 0.000s
I20250112 20:57:40.417284 16891 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:40.425253 16786 raft_consensus.cc:3054] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.426100 16875 consensus_queue.cc:1035] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:40.427568 16680 leader_election.cc:304] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:40.428503 16861 raft_consensus.cc:2798] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:40.428979 16861 raft_consensus.cc:695] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:40.429736 16861 consensus_queue.cc:237] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.431236 16635 raft_consensus.cc:2463] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:40.469884 16829 raft_consensus.cc:357] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.469921 16830 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:40.478871 16829 raft_consensus.cc:383] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.479084 16830 ts_tablet_manager.cc:1397] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.188s user 0.023s sys 0.041s
I20250112 20:57:40.479331 16829 raft_consensus.cc:738] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.471575 16828 tablet_bootstrap.cc:654] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.481611 16829 consensus_queue.cc:260] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:40.485239 17043 log.cc:926] Time spent T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d: Append to log took a long time: real 0.112s user 0.001s sys 0.001s
I20250112 20:57:40.491160 16829 ts_tablet_manager.cc:1428] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.148s user 0.006s sys 0.003s
I20250112 20:57:40.492565 16830 raft_consensus.cc:357] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.493294 16830 raft_consensus.cc:383] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.493561 16830 raft_consensus.cc:738] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.494065 16830 consensus_queue.cc:260] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.498005 16830 ts_tablet_manager.cc:1428] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20250112 20:57:40.498876 16830 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:40.509997 16891 consensus_queue.cc:1035] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:40.520519 16829 tablet_bootstrap.cc:492] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:40.530332 16828 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:40.530925 16828 ts_tablet_manager.cc:1397] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.249s user 0.048s sys 0.015s
I20250112 20:57:40.516799 17045 consensus_queue.cc:1035] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20250112 20:57:40.532801 16993 raft_consensus.cc:491] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.538141 16993 raft_consensus.cc:513] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.539901 16993 leader_election.cc:290] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.540220 16830 tablet_bootstrap.cc:654] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.541345 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:40.542074 16714 raft_consensus.cc:2388] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80770068c3a34f918765f54dbe3a94eb in current term 1: Already voted for candidate c29b243acf2b4f248606563fc96efc7d in this term.
I20250112 20:57:40.547994 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:40.550743 16830 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:40.551213 16830 ts_tablet_manager.cc:1397] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.004s
I20250112 20:57:40.553781 16678 leader_election.cc:304] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:40.555976 17042 raft_consensus.cc:2798] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:40.556296 16602 leader_election.cc:304] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:40.557425 16993 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.556423 17042 raft_consensus.cc:695] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:40.563723 16828 raft_consensus.cc:357] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.571658 16828 raft_consensus.cc:383] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.572005 16828 raft_consensus.cc:738] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.575112 16828 consensus_queue.cc:260] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.579572 16967 raft_consensus.cc:491] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.631691 17042 consensus_queue.cc:237] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.636706 16828 ts_tablet_manager.cc:1428] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.078s user 0.008s sys 0.001s
I20250112 20:57:40.639456 16830 raft_consensus.cc:357] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.640437 16830 raft_consensus.cc:383] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.640787 16830 raft_consensus.cc:738] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.641561 16830 consensus_queue.cc:260] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.645548 16828 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:40.583328 16967 raft_consensus.cc:513] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.659953 16830 ts_tablet_manager.cc:1428] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.108s user 0.009s sys 0.000s
I20250112 20:57:40.660878 16830 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
W20250112 20:57:40.673626 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:40.677197 16788 raft_consensus.cc:3054] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.687199 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9eab84e9c44a4aae8715e7735721d8" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:40.687583 16993 raft_consensus.cc:2743] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:40.677532 16967 leader_election.cc:290] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.691128 16891 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:40.692212 16680 leader_election.cc:343] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.711756 16830 tablet_bootstrap.cc:654] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.811861 16829 tablet_bootstrap.cc:654] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.815240 17013 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:40.826263 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9eab84e9c44a4aae8715e7735721d8" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:40.827970 16947 raft_consensus.cc:491] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.829422 16947 raft_consensus.cc:513] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.833072 16947 leader_election.cc:290] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.836133 16828 tablet_bootstrap.cc:654] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.837244 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305481b596b746b1a03df8d76e4057fb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:40.838337 17045 consensus_queue.cc:1035] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20250112 20:57:40.844902 16602 leader_election.cc:343] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.858070 16861 raft_consensus.cc:491] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.858608 16861 raft_consensus.cc:513] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.860782 17013 raft_consensus.cc:491] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.864303 17013 raft_consensus.cc:513] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.866269 17013 leader_election.cc:290] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:40.865535 16680 consensus_peers.cc:487] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:40.875206 16535 catalog_manager.cc:5526] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:40.877337 16535 catalog_manager.cc:5526] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:40.880220 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:40.880873 16709 raft_consensus.cc:2463] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:40.864804 16861 leader_election.cc:290] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.881752 17045 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20250112 20:57:40.886508 16602 leader_election.cc:304] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:40.897125 16781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:40.883838 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:40.904695 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305481b596b746b1a03df8d76e4057fb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:40.905373 16680 leader_election.cc:343] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:40.898422 16605 leader_election.cc:343] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.915309 16828 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:40.916947 16830 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
W20250112 20:57:40.916862 16605 leader_election.cc:343] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.917395 16830 ts_tablet_manager.cc:1397] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.257s user 0.051s sys 0.078s
I20250112 20:57:40.917389 16605 leader_election.cc:304] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:40.918751 16947 raft_consensus.cc:2743] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:40.918663 16829 tablet_bootstrap.cc:492] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:40.919591 16829 ts_tablet_manager.cc:1397] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.400s user 0.036s sys 0.038s
I20250112 20:57:40.920130 16828 ts_tablet_manager.cc:1397] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.271s user 0.012s sys 0.011s
I20250112 20:57:40.923508 16830 raft_consensus.cc:357] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.924240 16830 raft_consensus.cc:383] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.924556 16830 raft_consensus.cc:738] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.925417 17069 raft_consensus.cc:2798] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:40.925216 16830 consensus_queue.cc:260] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.925863 17069 raft_consensus.cc:491] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:40.926244 17069 raft_consensus.cc:3054] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:40.922515 16829 raft_consensus.cc:357] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.927345 16829 raft_consensus.cc:383] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:40.927561 16829 raft_consensus.cc:738] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:40.928929 16829 consensus_queue.cc:260] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.933284 16830 ts_tablet_manager.cc:1428] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
W20250112 20:57:40.935956 16678 leader_election.cc:343] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:40.936487 16678 leader_election.cc:304] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:40.937258 17042 raft_consensus.cc:2743] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:40.937232 16829 ts_tablet_manager.cc:1428] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.017s user 0.003s sys 0.002s
I20250112 20:57:40.938769 16829 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:40.941285 16830 tablet_bootstrap.cc:492] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:40.960181 17069 raft_consensus.cc:513] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.976584 16829 tablet_bootstrap.cc:654] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:40.979925 17069 leader_election.cc:290] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.983196 17069 raft_consensus.cc:491] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:40.985781 17069 raft_consensus.cc:513] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:40.998190 17069 leader_election.cc:290] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:40.999769 16892 consensus_queue.cc:1035] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:41.000212 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9eab84e9c44a4aae8715e7735721d8" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:41.002342 16830 tablet_bootstrap.cc:654] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:41.004002 16605 leader_election.cc:343] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.005213 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:41.007560 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9eab84e9c44a4aae8715e7735721d8" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.008162 16707 raft_consensus.cc:2463] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
W20250112 20:57:41.008451 16605 leader_election.cc:343] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.016652 16602 leader_election.cc:304] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:41.004885 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:41.018064 16947 raft_consensus.cc:2798] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:41.025349 16947 raft_consensus.cc:491] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:41.025729 16947 raft_consensus.cc:3054] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.029088 16637 raft_consensus.cc:2388] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 018d03f043b542b9b63491a77523968d in current term 1: Already voted for candidate 80770068c3a34f918765f54dbe3a94eb in this term.
I20250112 20:57:41.031276 16861 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:41.018317 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:41.033406 16711 raft_consensus.cc:3054] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.034322 16947 raft_consensus.cc:513] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.037083 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9eab84e9c44a4aae8715e7735721d8" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:41.037593 16707 raft_consensus.cc:3054] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.041836 16828 raft_consensus.cc:357] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.042474 16828 raft_consensus.cc:383] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.042729 16828 raft_consensus.cc:738] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.045759 16828 consensus_queue.cc:260] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.055307 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9eab84e9c44a4aae8715e7735721d8" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
W20250112 20:57:41.057389 16605 leader_election.cc:343] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.059926 16678 leader_election.cc:304] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
W20250112 20:57:41.061671 16751 consensus_peers.cc:487] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:41.068742 16829 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:41.069281 16829 ts_tablet_manager.cc:1397] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.131s user 0.019s sys 0.011s
I20250112 20:57:41.050926 16947 leader_election.cc:290] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.075224 16828 ts_tablet_manager.cc:1428] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.155s user 0.005s sys 0.002s
I20250112 20:57:41.084003 16828 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:41.084780 16830 tablet_bootstrap.cc:492] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:41.085186 16830 ts_tablet_manager.cc:1397] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.144s user 0.006s sys 0.008s
I20250112 20:57:41.089150 16711 raft_consensus.cc:2463] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:41.089643 16894 raft_consensus.cc:2743] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:41.088842 16830 raft_consensus.cc:357] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.091958 16830 raft_consensus.cc:383] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.092221 16830 raft_consensus.cc:738] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.092705 16830 consensus_queue.cc:260] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.094650 16829 raft_consensus.cc:357] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.094686 16602 leader_election.cc:304] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:41.095355 16829 raft_consensus.cc:383] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.095620 16829 raft_consensus.cc:738] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.096210 16829 consensus_queue.cc:260] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.097604 16829 ts_tablet_manager.cc:1428] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20250112 20:57:41.098242 16829 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:41.102408 16830 ts_tablet_manager.cc:1428] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20250112 20:57:41.103271 16830 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:41.103106 16707 raft_consensus.cc:2463] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:41.104097 16786 raft_consensus.cc:3054] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.110426 16891 raft_consensus.cc:2798] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:41.110889 16891 raft_consensus.cc:695] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:41.111846 16602 leader_election.cc:304] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:41.111665 16891 consensus_queue.cc:237] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.113343 16993 raft_consensus.cc:2798] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:41.113786 16993 raft_consensus.cc:695] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:41.114987 16894 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:41.114851 16993 consensus_queue.cc:237] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.140095 16828 tablet_bootstrap.cc:654] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.143828 16829 tablet_bootstrap.cc:654] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:41.168366 16678 consensus_peers.cc:487] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:41.187928 16861 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:41.194767 16830 tablet_bootstrap.cc:654] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.196278 16829 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:41.196805 16829 ts_tablet_manager.cc:1397] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.099s user 0.009s sys 0.002s
I20250112 20:57:41.199095 16829 raft_consensus.cc:357] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.199772 16829 raft_consensus.cc:383] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.200021 16829 raft_consensus.cc:738] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.200724 16829 consensus_queue.cc:260] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.202941 16829 ts_tablet_manager.cc:1428] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:57:41.203821 16829 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:41.203663 16535 catalog_manager.cc:5526] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:41.206076 16925 raft_consensus.cc:491] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.205335 16535 catalog_manager.cc:5526] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:41.206926 16925 raft_consensus.cc:513] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.210886 16925 leader_election.cc:290] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:41.212117 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.213474 16828 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:41.213939 16828 ts_tablet_manager.cc:1397] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.130s user 0.015s sys 0.027s
I20250112 20:57:41.215354 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:41.217070 16752 leader_election.cc:304] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:41.220779 16828 raft_consensus.cc:357] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.221699 16828 raft_consensus.cc:383] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.222018 16828 raft_consensus.cc:738] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.225173 16828 consensus_queue.cc:260] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.218612 16875 raft_consensus.cc:2743] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:41.229103 16830 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:41.229566 16830 ts_tablet_manager.cc:1397] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.126s user 0.022s sys 0.009s
I20250112 20:57:41.229943 16828 ts_tablet_manager.cc:1428] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20250112 20:57:41.230739 17042 raft_consensus.cc:491] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.231159 17042 raft_consensus.cc:513] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.235400 17042 leader_election.cc:290] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.236881 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:41.236899 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:41.240087 16828 tablet_bootstrap.cc:492] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:41.241094 16830 raft_consensus.cc:357] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.241632 16829 tablet_bootstrap.cc:654] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.241776 16830 raft_consensus.cc:383] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.242038 16830 raft_consensus.cc:738] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.243377 16678 leader_election.cc:304] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:41.245836 16892 raft_consensus.cc:2743] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:41.248129 16830 consensus_queue.cc:260] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.250922 16830 ts_tablet_manager.cc:1428] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.021s user 0.004s sys 0.000s
I20250112 20:57:41.252009 16830 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:41.258909 16788 raft_consensus.cc:3054] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.269075 16707 raft_consensus.cc:3054] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.291144 16828 tablet_bootstrap.cc:654] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.295146 16830 tablet_bootstrap.cc:654] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.296782 16829 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:41.297304 16829 ts_tablet_manager.cc:1397] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.094s user 0.021s sys 0.011s
I20250112 20:57:41.301779 16829 raft_consensus.cc:357] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.302498 16829 raft_consensus.cc:383] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.302771 16829 raft_consensus.cc:738] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.303341 16829 consensus_queue.cc:260] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.307266 16829 ts_tablet_manager.cc:1428] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.010s user 0.004s sys 0.001s
I20250112 20:57:41.308264 16829 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:41.321508 16830 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:41.321972 16830 ts_tablet_manager.cc:1397] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.070s user 0.004s sys 0.011s
I20250112 20:57:41.322227 16829 tablet_bootstrap.cc:654] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.324410 16830 raft_consensus.cc:357] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.325146 16830 raft_consensus.cc:383] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.325454 16830 raft_consensus.cc:738] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.326095 16830 consensus_queue.cc:260] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.328469 16830 ts_tablet_manager.cc:1428] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:57:41.330157 16830 tablet_bootstrap.cc:492] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:41.338033 16830 tablet_bootstrap.cc:654] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.340863 16788 raft_consensus.cc:3054] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.343348 16828 tablet_bootstrap.cc:492] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:41.343768 16828 ts_tablet_manager.cc:1397] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.104s user 0.009s sys 0.009s
I20250112 20:57:41.346367 16828 raft_consensus.cc:357] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.347148 16828 raft_consensus.cc:383] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.347748 16828 raft_consensus.cc:738] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.348446 16828 consensus_queue.cc:260] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.350666 16828 ts_tablet_manager.cc:1428] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20250112 20:57:41.352488 16828 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:41.370489 16830 tablet_bootstrap.cc:492] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:41.370841 16829 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:41.371047 16830 ts_tablet_manager.cc:1397] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.012s
I20250112 20:57:41.371351 16829 ts_tablet_manager.cc:1397] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.018s
I20250112 20:57:41.374080 16829 raft_consensus.cc:357] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.374768 16829 raft_consensus.cc:383] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.375030 16829 raft_consensus.cc:738] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.375545 16829 consensus_queue.cc:260] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.374815 16830 raft_consensus.cc:357] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.377537 16830 raft_consensus.cc:383] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.377830 16830 raft_consensus.cc:738] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.379016 16830 consensus_queue.cc:260] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.380657 16828 tablet_bootstrap.cc:654] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.381356 16830 ts_tablet_manager.cc:1428] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20250112 20:57:41.381829 16829 ts_tablet_manager.cc:1428] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20250112 20:57:41.383085 16829 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:41.385178 16830 tablet_bootstrap.cc:492] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:41.390673 16829 tablet_bootstrap.cc:654] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.393162 16830 tablet_bootstrap.cc:654] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.408401 16830 tablet_bootstrap.cc:492] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:41.408885 16830 ts_tablet_manager.cc:1397] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.004s
I20250112 20:57:41.411296 16830 raft_consensus.cc:357] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.412120 16830 raft_consensus.cc:383] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.412389 16830 raft_consensus.cc:738] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.413014 16830 consensus_queue.cc:260] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.415045 16830 ts_tablet_manager.cc:1428] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20250112 20:57:41.415939 16830 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
W20250112 20:57:41.486277 16751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:38087, user_credentials={real_user=slave}} blocked reactor thread for 63040us
I20250112 20:57:41.488122 16829 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:41.488529 16829 ts_tablet_manager.cc:1397] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.106s user 0.008s sys 0.010s
I20250112 20:57:41.491609 17013 raft_consensus.cc:491] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.492117 17013 raft_consensus.cc:513] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.498121 17013 leader_election.cc:290] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.499771 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:41.500954 16602 leader_election.cc:343] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.502810 16829 raft_consensus.cc:357] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.505478 16828 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:41.506031 16828 ts_tablet_manager.cc:1397] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.155s user 0.023s sys 0.016s
I20250112 20:57:41.506350 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:41.503521 16829 raft_consensus.cc:383] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.507875 16829 raft_consensus.cc:738] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.514981 16828 raft_consensus.cc:357] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.515550 16828 raft_consensus.cc:383] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.515791 16828 raft_consensus.cc:738] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.516290 16828 consensus_queue.cc:260] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.517637 16830 tablet_bootstrap.cc:654] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:41.518815 16605 leader_election.cc:343] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.520355 16891 raft_consensus.cc:491] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.520808 16891 raft_consensus.cc:513] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.521999 16828 ts_tablet_manager.cc:1428] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20250112 20:57:41.522922 16828 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:41.523455 16891 leader_election.cc:290] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.524197 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ffbc5c73802406893d78f89cf5c145c" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.525265 16707 raft_consensus.cc:2463] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:41.525377 16829 consensus_queue.cc:260] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.526441 16602 leader_election.cc:304] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:41.527447 17013 raft_consensus.cc:2798] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:41.527839 17013 raft_consensus.cc:491] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:41.528219 17013 raft_consensus.cc:3054] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.528443 16947 raft_consensus.cc:491] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.528937 16947 raft_consensus.cc:513] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.530087 16605 leader_election.cc:304] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:41.530707 16947 leader_election.cc:290] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.531168 16947 raft_consensus.cc:2743] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:41.578197 16829 ts_tablet_manager.cc:1428] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.089s user 0.000s sys 0.016s
I20250112 20:57:41.578495 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ffbc5c73802406893d78f89cf5c145c" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:41.579141 16786 raft_consensus.cc:2463] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:41.579350 16829 tablet_bootstrap.cc:492] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:41.580296 17005 raft_consensus.cc:491] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.582201 17005 raft_consensus.cc:513] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.590749 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9c04a57cac449e8efae9c7689ad690" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.592689 16711 raft_consensus.cc:2463] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:41.595642 16828 tablet_bootstrap.cc:654] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.579852 17013 raft_consensus.cc:513] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.597339 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ffbc5c73802406893d78f89cf5c145c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:41.599643 17005 leader_election.cc:290] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:41.600154 16634 raft_consensus.cc:2388] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c29b243acf2b4f248606563fc96efc7d in current term 1: Already voted for candidate 80770068c3a34f918765f54dbe3a94eb in this term.
I20250112 20:57:41.600898 16602 leader_election.cc:304] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:41.602741 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ffbc5c73802406893d78f89cf5c145c" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:41.603272 16709 raft_consensus.cc:3054] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.603431 16947 raft_consensus.cc:2798] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:41.603839 16947 raft_consensus.cc:491] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:41.604174 16947 raft_consensus.cc:3054] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.608057 16830 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:41.609794 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9c04a57cac449e8efae9c7689ad690" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:41.610006 16709 raft_consensus.cc:2463] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:41.610383 16830 ts_tablet_manager.cc:1397] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.195s user 0.011s sys 0.027s
I20250112 20:57:41.602087 17013 leader_election.cc:290] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.617045 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ffbc5c73802406893d78f89cf5c145c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.617820 16711 raft_consensus.cc:2388] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c29b243acf2b4f248606563fc96efc7d in current term 1: Already voted for candidate 80770068c3a34f918765f54dbe3a94eb in this term.
I20250112 20:57:41.618921 16751 leader_election.cc:304] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:41.619313 16829 tablet_bootstrap.cc:654] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.620314 16875 raft_consensus.cc:3054] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.622184 16947 raft_consensus.cc:513] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.623859 16947 leader_election.cc:290] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.626804 16875 raft_consensus.cc:2743] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:41.629056 16605 leader_election.cc:343] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:41.629993 16605 consensus_peers.cc:487] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:41.631906 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ffbc5c73802406893d78f89cf5c145c" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:41.638936 16786 raft_consensus.cc:2463] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:41.641974 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9c04a57cac449e8efae9c7689ad690" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:41.645018 16602 leader_election.cc:304] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:41.650130 16829 tablet_bootstrap.cc:492] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:41.650444 17102 raft_consensus.cc:2798] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:41.645650 16830 raft_consensus.cc:357] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.650943 16830 raft_consensus.cc:383] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.651223 16830 raft_consensus.cc:738] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.645593 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9c04a57cac449e8efae9c7689ad690" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:41.651834 16707 raft_consensus.cc:3054] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.651778 16830 consensus_queue.cc:260] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.655042 16830 ts_tablet_manager.cc:1428] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20250112 20:57:41.650583 16829 ts_tablet_manager.cc:1397] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.071s user 0.003s sys 0.014s
I20250112 20:57:41.656105 16830 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:41.662150 17102 raft_consensus.cc:695] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:41.663021 17102 consensus_queue.cc:237] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.663592 17069 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:41.664119 16828 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:41.664548 16828 ts_tablet_manager.cc:1397] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.142s user 0.018s sys 0.001s
I20250112 20:57:41.664763 17069 raft_consensus.cc:491] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.665207 17069 raft_consensus.cc:513] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.666165 16707 raft_consensus.cc:2463] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
W20250112 20:57:41.657480 16605 leader_election.cc:343] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.669981 16602 leader_election.cc:304] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:41.667168 17069 leader_election.cc:290] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.673696 16993 raft_consensus.cc:2798] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:41.671195 16829 raft_consensus.cc:357] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.672389 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10f3f7e0f74c9f988fa01e7843e961" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.677024 16829 raft_consensus.cc:383] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.679087 16829 raft_consensus.cc:738] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.679735 16829 consensus_queue.cc:260] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.680958 16712 raft_consensus.cc:2463] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:41.676911 16993 raft_consensus.cc:695] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:41.682291 16993 consensus_queue.cc:237] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.686655 16602 leader_election.cc:304] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:41.687986 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10f3f7e0f74c9f988fa01e7843e961" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:41.687354 17013 raft_consensus.cc:2798] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:41.692665 16891 raft_consensus.cc:491] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.693087 16891 raft_consensus.cc:513] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.694931 16891 leader_election.cc:290] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.692338 17013 raft_consensus.cc:491] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:41.695873 17013 raft_consensus.cc:3054] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.698189 16830 tablet_bootstrap.cc:654] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.679844 16828 raft_consensus.cc:357] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.699120 16828 raft_consensus.cc:383] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.699381 16828 raft_consensus.cc:738] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.699920 16828 consensus_queue.cc:260] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.701088 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:41.704916 16602 leader_election.cc:343] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:41.708328 16605 leader_election.cc:343] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.709791 16829 ts_tablet_manager.cc:1428] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.052s user 0.005s sys 0.001s
I20250112 20:57:41.712340 16828 ts_tablet_manager.cc:1428] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.047s user 0.006s sys 0.000s
I20250112 20:57:41.713754 16829 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:41.713976 16828 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
W20250112 20:57:41.710496 16605 consensus_peers.cc:487] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:41.716984 16875 raft_consensus.cc:491] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.711225 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:41.717871 16783 raft_consensus.cc:2463] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:41.717561 16875 raft_consensus.cc:513] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.721103 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1d0b3432b644821ba556f59a893d40a" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:41.722652 17069 consensus_queue.cc:1035] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250112 20:57:41.726343 16752 leader_election.cc:343] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.727389 16875 leader_election.cc:290] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:41.729619 16605 leader_election.cc:304] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:41.731824 16947 raft_consensus.cc:2798] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:41.728785 17013 raft_consensus.cc:513] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.740316 16947 raft_consensus.cc:491] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:41.750169 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1d0b3432b644821ba556f59a893d40a" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.753170 16712 raft_consensus.cc:2463] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:41.755570 16830 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:41.756008 16830 ts_tablet_manager.cc:1397] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.100s user 0.034s sys 0.006s
I20250112 20:57:41.748685 16947 raft_consensus.cc:3054] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.760066 16751 leader_election.cc:304] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:41.761741 17005 raft_consensus.cc:2798] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:41.762156 17005 raft_consensus.cc:491] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:41.762480 17005 raft_consensus.cc:3054] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.791714 16828 tablet_bootstrap.cc:654] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.791811 17013 leader_election.cc:290] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.789981 16947 raft_consensus.cc:513] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.795593 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10f3f7e0f74c9f988fa01e7843e961" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:41.796754 17005 raft_consensus.cc:513] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.794909 16830 raft_consensus.cc:357] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.801424 16830 raft_consensus.cc:383] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.801798 16830 raft_consensus.cc:738] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.797173 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10f3f7e0f74c9f988fa01e7843e961" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
W20250112 20:57:41.798187 16605 leader_election.cc:343] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.805028 16829 tablet_bootstrap.cc:654] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:41.804771 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1d0b3432b644821ba556f59a893d40a" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:41.804745 16709 raft_consensus.cc:3054] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.805826 17005 leader_election.cc:290] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:41.806294 16947 leader_election.cc:290] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:41.807407 16830 consensus_queue.cc:260] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.807085 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
W20250112 20:57:41.810703 16602 leader_election.cc:343] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:41.811470 16752 leader_election.cc:343] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.808830 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:41.813330 16783 raft_consensus.cc:3054] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.822963 16535 catalog_manager.cc:5526] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:41.824471 16535 catalog_manager.cc:5526] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:41.825646 16830 ts_tablet_manager.cc:1428] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.069s user 0.007s sys 0.002s
I20250112 20:57:41.826520 16830 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:41.829519 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1d0b3432b644821ba556f59a893d40a" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:41.830060 16712 raft_consensus.cc:3054] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.832489 16783 raft_consensus.cc:2463] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:41.833590 16605 leader_election.cc:304] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:41.834606 16947 raft_consensus.cc:2798] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:41.835053 16947 raft_consensus.cc:695] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:41.836366 16947 consensus_queue.cc:237] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.839660 16709 raft_consensus.cc:2463] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:41.840807 16602 leader_election.cc:304] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:41.842178 17069 raft_consensus.cc:2798] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:41.842614 17069 raft_consensus.cc:695] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:41.843329 17069 consensus_queue.cc:237] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.858639 16712 raft_consensus.cc:2463] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:41.860253 16751 leader_election.cc:304] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:41.861312 16925 raft_consensus.cc:2798] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:41.861773 16925 raft_consensus.cc:695] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:41.863054 16925 consensus_queue.cc:237] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.874497 16828 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:41.874954 16828 ts_tablet_manager.cc:1397] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.161s user 0.012s sys 0.015s
I20250112 20:57:41.880139 16875 raft_consensus.cc:491] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.880690 16875 raft_consensus.cc:513] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.887104 16875 leader_election.cc:290] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:41.909118 16829 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:41.909289 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305481b596b746b1a03df8d76e4057fb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:41.909988 16639 raft_consensus.cc:2463] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:41.909588 16829 ts_tablet_manager.cc:1397] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.196s user 0.009s sys 0.008s
I20250112 20:57:41.919852 16752 leader_election.cc:304] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:41.920755 16875 raft_consensus.cc:2798] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:41.921187 16875 raft_consensus.cc:491] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:41.921558 16875 raft_consensus.cc:3054] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.935482 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305481b596b746b1a03df8d76e4057fb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.956863 16829 raft_consensus.cc:357] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.957433 16830 tablet_bootstrap.cc:654] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:41.958668 16751 leader_election.cc:343] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.957701 16829 raft_consensus.cc:383] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.960148 16829 raft_consensus.cc:738] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.960765 16875 raft_consensus.cc:513] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.966987 16875 leader_election.cc:290] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:57:41.969504 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:41.971297 16829 consensus_queue.cc:260] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.972641 16828 raft_consensus.cc:357] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.973521 16828 raft_consensus.cc:383] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:41.973783 16828 raft_consensus.cc:738] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:41.974367 16828 consensus_queue.cc:260] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.977921 16828 ts_tablet_manager.cc:1428] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.103s user 0.013s sys 0.000s
I20250112 20:57:41.978765 16828 tablet_bootstrap.cc:492] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:41.982486 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305481b596b746b1a03df8d76e4057fb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:41.983115 16639 raft_consensus.cc:3054] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:41.985916 17013 raft_consensus.cc:491] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:41.986404 17013 raft_consensus.cc:513] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:41.987929 16829 ts_tablet_manager.cc:1428] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.077s user 0.007s sys 0.002s
I20250112 20:57:41.989114 16829 tablet_bootstrap.cc:492] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:41.991487 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23f9778f8494e11b9d942035720e363" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:41.992978 16605 leader_election.cc:343] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:41.993394 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23f9778f8494e11b9d942035720e363" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:41.993989 16712 raft_consensus.cc:2463] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:41.995019 16602 leader_election.cc:304] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:41.996729 17069 raft_consensus.cc:2798] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:41.988401 17013 leader_election.cc:290] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.003409 17069 raft_consensus.cc:491] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:42.010205 17069 raft_consensus.cc:3054] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.032528 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305481b596b746b1a03df8d76e4057fb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:42.037607 16830 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:42.037990 16830 ts_tablet_manager.cc:1397] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.212s user 0.030s sys 0.020s
I20250112 20:57:42.038618 16534 catalog_manager.cc:5526] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
W20250112 20:57:42.041822 16751 leader_election.cc:343] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.043388 16636 raft_consensus.cc:3054] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.054610 16830 raft_consensus.cc:357] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.055256 16830 raft_consensus.cc:383] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.055174 16947 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.055624 16830 raft_consensus.cc:738] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.056224 16830 consensus_queue.cc:260] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.058437 16830 ts_tablet_manager.cc:1428] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20250112 20:57:42.059247 16830 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:42.060938 16639 raft_consensus.cc:2463] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:42.062283 16828 tablet_bootstrap.cc:654] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.062985 16752 leader_election.cc:304] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:42.064816 16925 raft_consensus.cc:2798] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:42.066215 16925 raft_consensus.cc:695] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:42.052189 17069 raft_consensus.cc:513] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.074093 16925 consensus_queue.cc:237] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.077733 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23f9778f8494e11b9d942035720e363" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:42.078315 16707 raft_consensus.cc:3054] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.079759 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23f9778f8494e11b9d942035720e363" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:42.080797 16829 tablet_bootstrap.cc:654] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:42.088891 16605 leader_election.cc:343] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.090837 16707 raft_consensus.cc:2463] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:42.091828 16602 leader_election.cc:304] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.092645 16947 raft_consensus.cc:2798] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:42.093055 16947 raft_consensus.cc:695] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:42.094249 16947 consensus_queue.cc:237] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.076922 17069 leader_election.cc:290] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.120679 16830 tablet_bootstrap.cc:654] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.134972 16537 catalog_manager.cc:5526] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:42.143289 17013 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.145640 16535 catalog_manager.cc:5526] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:42.146857 16535 catalog_manager.cc:5526] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:42.161798 17013 consensus_queue.cc:1035] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.170614 16829 tablet_bootstrap.cc:492] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:42.171090 16829 ts_tablet_manager.cc:1397] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.182s user 0.003s sys 0.016s
I20250112 20:57:42.179036 16828 tablet_bootstrap.cc:492] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:42.179472 16828 ts_tablet_manager.cc:1397] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.201s user 0.017s sys 0.032s
I20250112 20:57:42.179669 16829 raft_consensus.cc:357] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.180347 16829 raft_consensus.cc:383] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.180624 16829 raft_consensus.cc:738] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.181169 16829 consensus_queue.cc:260] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.181703 16828 raft_consensus.cc:357] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.182304 16828 raft_consensus.cc:383] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.182518 16828 raft_consensus.cc:738] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.183156 16829 ts_tablet_manager.cc:1428] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.012s user 0.004s sys 0.001s
I20250112 20:57:42.183027 16828 consensus_queue.cc:260] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.183912 16829 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:42.185312 16828 ts_tablet_manager.cc:1428] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:57:42.186245 16828 tablet_bootstrap.cc:492] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:42.207947 17098 raft_consensus.cc:491] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.211763 17098 raft_consensus.cc:513] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.214026 17098 leader_election.cc:290] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.217216 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:42.218730 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.219367 16634 raft_consensus.cc:2463] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
W20250112 20:57:42.219522 16680 leader_election.cc:343] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.221247 16828 tablet_bootstrap.cc:654] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.228442 16678 leader_election.cc:304] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.229905 16892 raft_consensus.cc:2798] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:42.230263 16892 raft_consensus.cc:491] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:42.230563 16892 raft_consensus.cc:3054] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.235110 16829 tablet_bootstrap.cc:654] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.251386 16892 raft_consensus.cc:513] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.269116 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
W20250112 20:57:42.270951 16680 leader_election.cc:343] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.271911 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:42.272692 16635 raft_consensus.cc:3054] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.274381 16892 leader_election.cc:290] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.276320 16534 catalog_manager.cc:5526] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:42.286810 16830 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:42.287218 16830 ts_tablet_manager.cc:1397] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.228s user 0.010s sys 0.016s
I20250112 20:57:42.305228 16830 raft_consensus.cc:357] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.305935 16830 raft_consensus.cc:383] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.306169 16830 raft_consensus.cc:738] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.307523 16830 consensus_queue.cc:260] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.305651 16786 raft_consensus.cc:3054] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.310309 16830 ts_tablet_manager.cc:1428] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20250112 20:57:42.319758 16830 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:42.312426 16829 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:42.320421 16829 ts_tablet_manager.cc:1397] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.137s user 0.011s sys 0.012s
I20250112 20:57:42.313299 16828 tablet_bootstrap.cc:492] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:42.321051 16828 ts_tablet_manager.cc:1397] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.135s user 0.011s sys 0.013s
I20250112 20:57:42.318917 16635 raft_consensus.cc:2463] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:42.329654 16678 leader_election.cc:304] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.330632 16892 raft_consensus.cc:2798] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:42.331864 16892 raft_consensus.cc:695] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:42.332710 16892 consensus_queue.cc:237] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.338296 16925 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:42.339731 16602 consensus_peers.cc:487] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:42.342092 17045 raft_consensus.cc:491] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.342512 17045 raft_consensus.cc:513] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.343994 16947 consensus_queue.cc:1035] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.344377 17045 leader_election.cc:290] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:42.344949 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.345598 16635 raft_consensus.cc:2463] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:42.346215 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:42.346525 16752 leader_election.cc:304] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:42.347291 17045 raft_consensus.cc:2798] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:42.347718 17045 raft_consensus.cc:491] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20250112 20:57:42.347781 16751 leader_election.cc:343] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.348182 17045 raft_consensus.cc:3054] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.353992 16947 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:42.359915 16828 raft_consensus.cc:357] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.366180 16828 raft_consensus.cc:383] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.369506 16828 raft_consensus.cc:738] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.370729 17045 raft_consensus.cc:513] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.361708 16829 raft_consensus.cc:357] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.376776 16829 raft_consensus.cc:383] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.377071 16829 raft_consensus.cc:738] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.377648 16829 consensus_queue.cc:260] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.379330 16829 ts_tablet_manager.cc:1428] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.059s user 0.003s sys 0.000s
I20250112 20:57:42.380081 16829 tablet_bootstrap.cc:492] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:42.381773 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:42.382416 16634 raft_consensus.cc:3054] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.381727 16537 catalog_manager.cc:5526] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:42.371235 16828 consensus_queue.cc:260] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.359910 16830 tablet_bootstrap.cc:654] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.384821 16828 ts_tablet_manager.cc:1428] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.064s user 0.005s sys 0.002s
I20250112 20:57:42.368640 16635 raft_consensus.cc:3054] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.386041 16828 tablet_bootstrap.cc:492] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:42.389791 17045 leader_election.cc:290] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:42.390730 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
W20250112 20:57:42.392289 16751 leader_election.cc:343] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.402137 16634 raft_consensus.cc:2463] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:42.403268 16752 leader_election.cc:304] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:42.403501 16925 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.404734 16925 raft_consensus.cc:2798] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:42.405185 16925 raft_consensus.cc:695] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:42.406538 16925 consensus_queue.cc:237] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.410991 16828 tablet_bootstrap.cc:654] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.411851 16829 tablet_bootstrap.cc:654] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.413189 17013 consensus_queue.cc:1035] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.426609 16830 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:42.426975 16830 ts_tablet_manager.cc:1397] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.107s user 0.004s sys 0.010s
I20250112 20:57:42.434213 16829 tablet_bootstrap.cc:492] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:42.434804 16829 ts_tablet_manager.cc:1397] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.055s user 0.011s sys 0.007s
W20250112 20:57:42.437152 16605 consensus_peers.cc:487] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:42.446049 16830 raft_consensus.cc:357] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.447050 16830 raft_consensus.cc:383] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.447309 16830 raft_consensus.cc:738] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.452976 16828 tablet_bootstrap.cc:492] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:42.453419 16828 ts_tablet_manager.cc:1397] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.068s user 0.009s sys 0.014s
I20250112 20:57:42.447978 16830 consensus_queue.cc:260] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.457502 16828 raft_consensus.cc:357] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.459070 16828 raft_consensus.cc:383] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.459275 16828 raft_consensus.cc:738] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.459805 16828 consensus_queue.cc:260] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.462308 16828 ts_tablet_manager.cc:1428] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20250112 20:57:42.463861 16828 tablet_bootstrap.cc:492] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:42.469108 16829 raft_consensus.cc:357] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.469236 16925 consensus_queue.cc:1035] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.469872 16829 raft_consensus.cc:383] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.470162 16829 raft_consensus.cc:738] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.470899 16829 consensus_queue.cc:260] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.473040 16829 ts_tablet_manager.cc:1428] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.038s user 0.006s sys 0.000s
I20250112 20:57:42.473821 16829 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:42.476868 16830 ts_tablet_manager.cc:1428] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.050s user 0.008s sys 0.000s
I20250112 20:57:42.477711 16830 tablet_bootstrap.cc:492] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:42.485044 16993 raft_consensus.cc:491] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.485515 16993 raft_consensus.cc:513] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.486176 16925 raft_consensus.cc:491] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.486609 16925 raft_consensus.cc:513] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.487622 16993 leader_election.cc:290] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.489380 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.490172 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:42.489382 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:42.491452 16751 leader_election.cc:343] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:42.491986 16752 leader_election.cc:343] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.492395 16752 leader_election.cc:304] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:42.493243 16875 raft_consensus.cc:2743] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:42.493527 16605 leader_election.cc:343] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.494071 16828 tablet_bootstrap.cc:654] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.500659 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:42.496322 16925 leader_election.cc:290] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:57:42.502846 16602 leader_election.cc:343] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.503297 16602 leader_election.cc:304] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.504405 16993 raft_consensus.cc:2743] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:42.506901 16830 tablet_bootstrap.cc:654] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.511544 16537 catalog_manager.cc:5526] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:42.540555 16829 tablet_bootstrap.cc:654] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:42.550343 16751 consensus_peers.cc:487] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:42.557022 16830 tablet_bootstrap.cc:492] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:42.557533 16830 ts_tablet_manager.cc:1397] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.080s user 0.007s sys 0.005s
I20250112 20:57:42.560285 16830 raft_consensus.cc:357] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.561158 16830 raft_consensus.cc:383] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.561312 16828 tablet_bootstrap.cc:492] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:42.561472 16830 raft_consensus.cc:738] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.561787 16828 ts_tablet_manager.cc:1397] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.098s user 0.008s sys 0.006s
I20250112 20:57:42.562295 16830 consensus_queue.cc:260] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.564249 16828 raft_consensus.cc:357] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.564962 16828 raft_consensus.cc:383] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.565248 16828 raft_consensus.cc:738] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.565883 16828 consensus_queue.cc:260] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.566898 16829 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:42.567306 16829 ts_tablet_manager.cc:1397] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.094s user 0.011s sys 0.006s
I20250112 20:57:42.568264 16828 ts_tablet_manager.cc:1428] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:57:42.569147 16828 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:42.573204 16830 ts_tablet_manager.cc:1428] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20250112 20:57:42.574733 16829 raft_consensus.cc:357] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.576184 16829 raft_consensus.cc:383] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.576501 16829 raft_consensus.cc:738] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.575915 16830 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:42.582460 16829 consensus_queue.cc:260] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.584694 16829 ts_tablet_manager.cc:1428] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.017s user 0.007s sys 0.001s
I20250112 20:57:42.584571 17069 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.587869 16829 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:42.586220 16830 tablet_bootstrap.cc:654] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.617946 17098 raft_consensus.cc:491] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.619428 16828 tablet_bootstrap.cc:654] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.619040 17098 raft_consensus.cc:513] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.621132 17098 leader_election.cc:290] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.622126 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:42.624800 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.628181 16680 leader_election.cc:304] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.629834 17098 raft_consensus.cc:2743] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:42.630892 16605 consensus_peers.cc:487] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:42.636176 16828 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:42.636687 16828 ts_tablet_manager.cc:1397] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.068s user 0.009s sys 0.003s
I20250112 20:57:42.639062 16828 raft_consensus.cc:357] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.640249 16828 raft_consensus.cc:383] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.647282 16828 raft_consensus.cc:738] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.646536 16830 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:42.637369 16829 tablet_bootstrap.cc:654] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.648211 16830 ts_tablet_manager.cc:1397] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.072s user 0.015s sys 0.000s
I20250112 20:57:42.651022 16830 raft_consensus.cc:357] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.652623 16830 raft_consensus.cc:383] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.652952 16830 raft_consensus.cc:738] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.653640 16830 consensus_queue.cc:260] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.655948 16830 ts_tablet_manager.cc:1428] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20250112 20:57:42.651875 16828 consensus_queue.cc:260] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.658033 16828 ts_tablet_manager.cc:1428] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20250112 20:57:42.659046 16830 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:42.664758 16828 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:42.665567 16788 raft_consensus.cc:3054] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.688988 16828 tablet_bootstrap.cc:654] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.693158 16830 tablet_bootstrap.cc:654] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.694157 16829 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:42.694636 16829 ts_tablet_manager.cc:1397] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.107s user 0.016s sys 0.010s
I20250112 20:57:42.705570 16829 raft_consensus.cc:357] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.706265 16829 raft_consensus.cc:383] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.706521 16829 raft_consensus.cc:738] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.707087 16829 consensus_queue.cc:260] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.708810 16828 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:42.709241 16828 ts_tablet_manager.cc:1397] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.045s user 0.016s sys 0.001s
I20250112 20:57:42.711768 16828 raft_consensus.cc:357] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.712430 16828 raft_consensus.cc:383] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.712777 16829 ts_tablet_manager.cc:1428] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20250112 20:57:42.712715 16828 raft_consensus.cc:738] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.714196 16829 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:42.713963 16828 consensus_queue.cc:260] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.720739 16829 tablet_bootstrap.cc:654] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.724689 16828 ts_tablet_manager.cc:1428] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20250112 20:57:42.731531 16714 raft_consensus.cc:3054] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.741526 16875 raft_consensus.cc:491] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.741963 16875 raft_consensus.cc:513] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.743589 16875 leader_election.cc:290] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:42.744051 16828 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:42.744803 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9eab84e9c44a4aae8715e7735721d8" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:42.745206 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b9eab84e9c44a4aae8715e7735721d8" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.747932 16751 leader_election.cc:304] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:42.749433 17067 raft_consensus.cc:2743] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:42.754849 16829 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:42.755290 16829 ts_tablet_manager.cc:1397] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.011s
I20250112 20:57:42.758464 16830 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:42.758872 16830 ts_tablet_manager.cc:1397] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.100s user 0.011s sys 0.002s
I20250112 20:57:42.757630 16829 raft_consensus.cc:357] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.761081 16829 raft_consensus.cc:383] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.761361 16829 raft_consensus.cc:738] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.761937 16829 consensus_queue.cc:260] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.762960 16828 tablet_bootstrap.cc:654] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.764045 16829 ts_tablet_manager.cc:1428] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20250112 20:57:42.765522 16830 raft_consensus.cc:357] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.766330 16830 raft_consensus.cc:383] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.766667 16830 raft_consensus.cc:738] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.767374 16830 consensus_queue.cc:260] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.769516 16993 raft_consensus.cc:491] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.769922 16993 raft_consensus.cc:513] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.771814 16993 leader_election.cc:290] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.773358 16894 raft_consensus.cc:491] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.773823 16894 raft_consensus.cc:513] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.775112 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:42.775535 16894 leader_election.cc:290] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:42.776371 16602 leader_election.cc:343] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.776623 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:42.776963 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.777202 16788 raft_consensus.cc:2463] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
W20250112 20:57:42.778157 16678 leader_election.cc:343] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.780170 16605 leader_election.cc:304] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:42.781216 16993 raft_consensus.cc:2798] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:42.781570 16993 raft_consensus.cc:491] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:42.781891 16993 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.783248 16894 consensus_queue.cc:1035] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.784443 16894 raft_consensus.cc:491] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.784869 16894 raft_consensus.cc:513] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.780202 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:42.786388 16783 raft_consensus.cc:2463] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:42.787536 16680 leader_election.cc:304] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:42.789121 17098 raft_consensus.cc:2798] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:42.789472 17098 raft_consensus.cc:491] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:42.789798 17098 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.791882 16894 leader_election.cc:290] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.795060 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03cbd278697d4f6780211b21028f6d18" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.795670 16635 raft_consensus.cc:2463] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:42.796780 16678 leader_election.cc:304] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:42.799048 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03cbd278697d4f6780211b21028f6d18" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:42.794178 16993 raft_consensus.cc:513] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:42.800307 16680 leader_election.cc:343] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.800040 16830 ts_tablet_manager.cc:1428] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.041s user 0.006s sys 0.002s
I20250112 20:57:42.800668 16894 raft_consensus.cc:2798] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:42.804165 16894 raft_consensus.cc:491] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:42.804625 16894 raft_consensus.cc:3054] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.809052 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:42.809376 16829 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
W20250112 20:57:42.810266 16602 leader_election.cc:343] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.809309 16788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:42.813028 16788 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.816141 16830 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:42.802896 17098 raft_consensus.cc:513] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.820770 16788 raft_consensus.cc:2463] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:42.822356 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
W20250112 20:57:42.823652 16678 leader_election.cc:343] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.825539 16605 leader_election.cc:304] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d
I20250112 20:57:42.826114 16829 tablet_bootstrap.cc:654] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.826264 17102 raft_consensus.cc:2798] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:42.826761 17102 raft_consensus.cc:695] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:42.828109 17102 consensus_queue.cc:237] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.833034 16783 raft_consensus.cc:3054] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.835126 16993 leader_election.cc:290] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.840993 17098 leader_election.cc:290] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.842308 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
W20250112 20:57:42.842733 16680 consensus_peers.cc:487] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:42.842878 16784 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.832096 16894 raft_consensus.cc:513] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.856536 16784 raft_consensus.cc:2463] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:42.857630 16680 leader_election.cc:304] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters: 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:42.859422 17098 raft_consensus.cc:2798] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:42.860112 16828 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:42.860574 16828 ts_tablet_manager.cc:1397] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.135s user 0.019s sys 0.000s
I20250112 20:57:42.861590 17098 raft_consensus.cc:695] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:42.862732 17098 consensus_queue.cc:237] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.865478 16830 tablet_bootstrap.cc:654] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:42.868276 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03cbd278697d4f6780211b21028f6d18" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:42.868539 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03cbd278697d4f6780211b21028f6d18" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:42.868902 16636 raft_consensus.cc:3054] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:57:42.869715 16680 leader_election.cc:343] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.870822 16894 leader_election.cc:290] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.886539 16636 raft_consensus.cc:2463] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:42.888056 16678 leader_election.cc:304] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.889509 16892 raft_consensus.cc:2798] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:42.889964 16892 raft_consensus.cc:695] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:42.890748 16892 consensus_queue.cc:237] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.894380 17069 raft_consensus.cc:491] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.894873 17069 raft_consensus.cc:513] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.898208 17069 leader_election.cc:290] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.899210 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2820c3d5974dfcb2230e18411e6493" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:42.900425 16602 leader_election.cc:343] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.874063 16534 catalog_manager.cc:5526] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:42.902740 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2820c3d5974dfcb2230e18411e6493" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:42.903909 16605 leader_election.cc:343] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.904278 16605 leader_election.cc:304] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.904994 17069 raft_consensus.cc:2743] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:42.912710 16892 raft_consensus.cc:491] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.910503 16828 raft_consensus.cc:357] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.913157 16892 raft_consensus.cc:513] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.914700 16828 raft_consensus.cc:383] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.915403 16828 raft_consensus.cc:738] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.916359 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.916843 16892 leader_election.cc:290] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.917281 16634 raft_consensus.cc:2388] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 018d03f043b542b9b63491a77523968d in current term 1: Already voted for candidate c29b243acf2b4f248606563fc96efc7d in this term.
I20250112 20:57:42.915388 16534 catalog_manager.cc:5526] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:42.918715 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:42.920017 16680 leader_election.cc:304] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.922209 16892 raft_consensus.cc:3054] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.930330 16783 raft_consensus.cc:3054] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:42.937837 16537 catalog_manager.cc:5526] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:42.948235 16828 consensus_queue.cc:260] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.952497 16828 ts_tablet_manager.cc:1428] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.092s user 0.005s sys 0.000s
I20250112 20:57:42.955384 16830 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:42.956683 16830 ts_tablet_manager.cc:1397] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.153s user 0.013s sys 0.003s
I20250112 20:57:42.958029 16829 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:42.958442 16829 ts_tablet_manager.cc:1397] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.149s user 0.015s sys 0.000s
I20250112 20:57:42.959780 16830 raft_consensus.cc:357] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.961418 16828 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:42.968900 16830 raft_consensus.cc:383] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.969861 16830 raft_consensus.cc:738] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.960413 16829 raft_consensus.cc:357] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.960398 16892 raft_consensus.cc:2743] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:42.972478 16829 raft_consensus.cc:383] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:42.972823 16829 raft_consensus.cc:738] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:42.973484 16829 consensus_queue.cc:260] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.977703 16875 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:42.976482 16830 consensus_queue.cc:260] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.982473 16830 ts_tablet_manager.cc:1428] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
I20250112 20:57:42.989904 17156 raft_consensus.cc:491] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:42.990350 17156 raft_consensus.cc:513] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:42.992177 17156 leader_election.cc:290] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:42.993414 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:42.994082 16634 raft_consensus.cc:2463] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:42.995149 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:42.996732 16680 leader_election.cc:343] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:42.997426 16829 ts_tablet_manager.cc:1428] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.039s user 0.005s sys 0.004s
I20250112 20:57:42.998131 16829 tablet_bootstrap.cc:492] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:42.997895 16678 leader_election.cc:304] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:42.984530 16830 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:43.004696 16892 raft_consensus.cc:2798] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:43.012176 16829 tablet_bootstrap.cc:654] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.015395 16828 tablet_bootstrap.cc:654] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.012450 16892 raft_consensus.cc:491] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:43.016743 16892 raft_consensus.cc:3054] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:57:43.018812 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:43.031807 17067 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:43.070609 16828 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:43.074011 16828 ts_tablet_manager.cc:1397] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.113s user 0.007s sys 0.017s
I20250112 20:57:43.073478 16829 tablet_bootstrap.cc:492] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:43.076946 16829 ts_tablet_manager.cc:1397] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.079s user 0.011s sys 0.013s
I20250112 20:57:43.078408 16828 raft_consensus.cc:357] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.075526 16830 tablet_bootstrap.cc:654] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.079152 16829 raft_consensus.cc:357] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.079849 16829 raft_consensus.cc:383] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.080129 16829 raft_consensus.cc:738] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.081032 16892 raft_consensus.cc:513] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.079191 16828 raft_consensus.cc:383] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.082041 16828 raft_consensus.cc:738] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.082875 16892 leader_election.cc:290] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.082659 16828 consensus_queue.cc:260] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.083518 16829 consensus_queue.cc:260] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.085278 16828 ts_tablet_manager.cc:1428] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20250112 20:57:43.085958 16829 ts_tablet_manager.cc:1428] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20250112 20:57:43.086215 16828 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:43.086843 16829 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:43.089861 16830 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:43.135905 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:43.137786 16636 raft_consensus.cc:3054] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.138207 17170 raft_consensus.cc:491] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.139379 17170 raft_consensus.cc:513] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.150916 17170 leader_election.cc:290] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.152635 16636 raft_consensus.cc:2463] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:43.182298 16828 tablet_bootstrap.cc:654] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.185606 16875 raft_consensus.cc:491] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.090344 16830 ts_tablet_manager.cc:1397] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.106s user 0.009s sys 0.002s
I20250112 20:57:43.186478 16875 raft_consensus.cc:513] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.189046 16875 leader_election.cc:290] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:43.182359 16829 tablet_bootstrap.cc:654] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.193820 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:43.194434 16714 raft_consensus.cc:2463] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:43.194589 16830 raft_consensus.cc:357] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.195310 16830 raft_consensus.cc:383] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.195602 16830 raft_consensus.cc:738] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.195715 16751 leader_election.cc:304] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:43.199091 17067 raft_consensus.cc:2798] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:43.199458 17067 raft_consensus.cc:491] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:43.199787 17067 raft_consensus.cc:3054] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.196318 16830 consensus_queue.cc:260] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.220082 16830 ts_tablet_manager.cc:1428] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.033s user 0.005s sys 0.003s
I20250112 20:57:43.233371 16830 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:43.250188 16678 leader_election.cc:304] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:43.251354 17156 raft_consensus.cc:2798] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:43.251893 17156 raft_consensus.cc:695] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:43.253221 17156 consensus_queue.cc:237] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.258746 16925 raft_consensus.cc:491] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.259338 16925 raft_consensus.cc:513] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.261200 16925 leader_election.cc:290] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:43.272253 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:43.283757 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:43.284039 17067 raft_consensus.cc:513] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.286716 16712 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:57:43.290156 16678 leader_election.cc:343] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.290776 16829 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:43.291941 16829 ts_tablet_manager.cc:1397] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.205s user 0.006s sys 0.013s
I20250112 20:57:43.294773 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:43.299489 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:43.302685 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:43.303506 16786 raft_consensus.cc:2388] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 018d03f043b542b9b63491a77523968d in current term 1: Already voted for candidate c29b243acf2b4f248606563fc96efc7d in this term.
I20250112 20:57:43.305342 17067 leader_election.cc:290] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:43.306979 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:43.307703 16709 raft_consensus.cc:3054] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.315400 16891 raft_consensus.cc:491] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.315976 16891 raft_consensus.cc:513] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.318399 16891 leader_election.cc:290] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.319101 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2477026d8d4fac945714c894cca485" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:43.320392 16602 leader_election.cc:343] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.323519 16534 catalog_manager.cc:5526] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:43.328115 16891 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:43.332158 16828 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:43.330122 16786 raft_consensus.cc:3054] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.333096 16829 raft_consensus.cc:357] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.336174 16829 raft_consensus.cc:383] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.336390 16829 raft_consensus.cc:738] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
W20250112 20:57:43.338402 16680 leader_election.cc:343] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.338223 16829 consensus_queue.cc:260] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.339497 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:43.344309 16828 ts_tablet_manager.cc:1397] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.258s user 0.015s sys 0.015s
I20250112 20:57:43.348508 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2477026d8d4fac945714c894cca485" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:43.349392 16830 tablet_bootstrap.cc:654] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.345948 16680 leader_election.cc:304] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:43.351594 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:43.361028 16829 ts_tablet_manager.cc:1428] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.069s user 0.005s sys 0.001s
W20250112 20:57:43.361881 16605 leader_election.cc:343] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.363180 16891 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:43.362258 16605 leader_election.cc:304] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
W20250112 20:57:43.365219 16752 leader_election.cc:343] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.368826 16891 raft_consensus.cc:2743] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:43.361495 16709 raft_consensus.cc:2463] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:43.381381 16829 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:43.382052 16894 raft_consensus.cc:2743] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:43.413184 16828 raft_consensus.cc:357] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.413959 16828 raft_consensus.cc:383] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.414248 16828 raft_consensus.cc:738] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.424991 16828 consensus_queue.cc:260] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.430861 16828 ts_tablet_manager.cc:1428] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.086s user 0.000s sys 0.007s
I20250112 20:57:43.432039 16828 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:43.439226 16829 tablet_bootstrap.cc:654] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.440343 16751 leader_election.cc:304] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
W20250112 20:57:43.437449 16752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:41463, user_credentials={real_user=slave}} blocked reactor thread for 56265.1us
W20250112 20:57:43.441836 16752 leader_election.cc:343] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:43.446749 16678 consensus_peers.cc:487] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:43.447567 16752 leader_election.cc:304] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:43.448246 17067 raft_consensus.cc:2798] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:43.453123 17067 raft_consensus.cc:695] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:43.454296 17067 consensus_queue.cc:237] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:43.459151 16680 consensus_peers.cc:487] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:43.475047 16830 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:43.475664 16830 ts_tablet_manager.cc:1397] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.254s user 0.017s sys 0.017s
I20250112 20:57:43.491228 16830 raft_consensus.cc:357] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.491993 16830 raft_consensus.cc:383] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.492297 16830 raft_consensus.cc:738] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.501999 16894 consensus_queue.cc:1035] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20250112 20:57:43.511250 16894 consensus_queue.cc:1035] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:43.519431 16828 tablet_bootstrap.cc:654] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.508343 16830 consensus_queue.cc:260] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.521617 16925 raft_consensus.cc:2743] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:43.525432 16709 raft_consensus.cc:3054] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.554813 16830 ts_tablet_manager.cc:1428] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.079s user 0.008s sys 0.001s
I20250112 20:57:43.551046 16537 catalog_manager.cc:5526] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:43.559968 16830 tablet_bootstrap.cc:492] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:43.581990 16829 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:43.584503 16828 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:43.584947 16828 ts_tablet_manager.cc:1397] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.153s user 0.009s sys 0.010s
I20250112 20:57:43.583292 16830 tablet_bootstrap.cc:654] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.587214 16828 raft_consensus.cc:357] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.587901 16828 raft_consensus.cc:383] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.588145 16828 raft_consensus.cc:738] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.584537 16829 ts_tablet_manager.cc:1397] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.203s user 0.006s sys 0.010s
I20250112 20:57:43.597074 16829 raft_consensus.cc:357] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.597790 16829 raft_consensus.cc:383] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.598078 16829 raft_consensus.cc:738] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.598730 16829 consensus_queue.cc:260] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.600018 16828 consensus_queue.cc:260] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.602946 16892 raft_consensus.cc:491] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c29b243acf2b4f248606563fc96efc7d)
I20250112 20:57:43.603262 16829 ts_tablet_manager.cc:1428] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.014s user 0.002s sys 0.003s
I20250112 20:57:43.603456 16892 raft_consensus.cc:513] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.605571 16828 ts_tablet_manager.cc:1428] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20250112 20:57:43.606832 16829 tablet_bootstrap.cc:492] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:43.607739 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305481b596b746b1a03df8d76e4057fb" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:43.609017 16828 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:43.612087 16892 leader_election.cc:290] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.617246 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "305481b596b746b1a03df8d76e4057fb" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:43.624033 16828 tablet_bootstrap.cc:654] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.625164 16680 leader_election.cc:304] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:43.626157 16892 raft_consensus.cc:2743] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:57:43.625161 16829 tablet_bootstrap.cc:654] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.642108 16829 tablet_bootstrap.cc:492] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:43.642556 16829 ts_tablet_manager.cc:1397] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.036s user 0.011s sys 0.003s
I20250112 20:57:43.650956 16828 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:43.651479 16828 ts_tablet_manager.cc:1397] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.043s user 0.009s sys 0.004s
I20250112 20:57:43.654384 16828 raft_consensus.cc:357] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.655016 16828 raft_consensus.cc:383] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.655267 16828 raft_consensus.cc:738] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.656684 16829 raft_consensus.cc:357] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.657393 16829 raft_consensus.cc:383] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.657646 16829 raft_consensus.cc:738] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.658223 16829 consensus_queue.cc:260] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:43.660132 17192 log.cc:926] Time spent T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20250112 20:57:43.660815 16828 consensus_queue.cc:260] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.662968 16828 ts_tablet_manager.cc:1428] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.011s user 0.006s sys 0.001s
I20250112 20:57:43.660406 16829 ts_tablet_manager.cc:1428] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.018s user 0.002s sys 0.003s
I20250112 20:57:43.664374 16829 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:43.670456 16829 tablet_bootstrap.cc:654] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.680792 17067 raft_consensus.cc:491] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.681332 17067 raft_consensus.cc:513] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.683701 17067 leader_election.cc:290] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:43.685355 16828 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:43.686092 16830 tablet_bootstrap.cc:492] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:43.686548 16830 ts_tablet_manager.cc:1397] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.127s user 0.011s sys 0.000s
I20250112 20:57:43.687778 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:43.688776 16634 raft_consensus.cc:2388] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c29b243acf2b4f248606563fc96efc7d in current term 1: Already voted for candidate 018d03f043b542b9b63491a77523968d in this term.
I20250112 20:57:43.692375 16830 raft_consensus.cc:357] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.693058 16830 raft_consensus.cc:383] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.693289 16830 raft_consensus.cc:738] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.693305 16828 tablet_bootstrap.cc:654] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.693854 16830 consensus_queue.cc:260] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.696552 16830 ts_tablet_manager.cc:1428] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20250112 20:57:43.706988 17069 raft_consensus.cc:491] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.707583 17069 raft_consensus.cc:513] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.710177 17069 leader_election.cc:290] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.710783 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:43.711412 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f021c2fa4d3143ff899283a33cb24538" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:43.718163 16830 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
W20250112 20:57:43.718873 16605 leader_election.cc:343] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.719661 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f021c2fa4d3143ff899283a33cb24538" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:43.720285 16712 raft_consensus.cc:2463] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:43.724639 16602 leader_election.cc:304] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:43.725425 17069 raft_consensus.cc:2798] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:43.725898 17069 raft_consensus.cc:491] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:43.726300 17069 raft_consensus.cc:3054] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.712881 16751 leader_election.cc:304] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:43.728729 17067 raft_consensus.cc:3054] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.738819 16892 raft_consensus.cc:491] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.739315 16892 raft_consensus.cc:513] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.741813 16892 leader_election.cc:290] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.742398 16875 raft_consensus.cc:491] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.742821 16875 raft_consensus.cc:513] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.744460 16875 leader_election.cc:290] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:43.737728 17069 raft_consensus.cc:513] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.748530 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f021c2fa4d3143ff899283a33cb24538" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:43.749285 16709 raft_consensus.cc:3054] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.752971 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2820c3d5974dfcb2230e18411e6493" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:43.757176 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2820c3d5974dfcb2230e18411e6493" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:43.757814 16637 raft_consensus.cc:2463] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
W20250112 20:57:43.759941 16680 leader_election.cc:343] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.761549 16678 leader_election.cc:304] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:43.762806 17156 raft_consensus.cc:2798] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:43.763131 17156 raft_consensus.cc:491] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:43.763447 17156 raft_consensus.cc:3054] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.774940 16709 raft_consensus.cc:2463] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:43.776341 16602 leader_election.cc:304] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:43.778002 16993 raft_consensus.cc:2798] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:43.778702 16993 raft_consensus.cc:695] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:43.779345 16783 raft_consensus.cc:3054] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.779667 16993 consensus_queue.cc:237] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.761574 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23f9778f8494e11b9d942035720e363" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:43.788455 16828 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:43.789072 16828 ts_tablet_manager.cc:1397] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.104s user 0.014s sys 0.001s
I20250112 20:57:43.793004 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23f9778f8494e11b9d942035720e363" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:43.798238 16828 raft_consensus.cc:357] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.801870 17069 leader_election.cc:290] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.802503 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f021c2fa4d3143ff899283a33cb24538" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:43.804399 16830 tablet_bootstrap.cc:654] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:43.804790 16605 leader_election.cc:343] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.810407 16785 raft_consensus.cc:3054] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.799161 16828 raft_consensus.cc:383] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.811976 16828 raft_consensus.cc:738] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.813056 16828 consensus_queue.cc:260] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.815953 16828 ts_tablet_manager.cc:1428] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20250112 20:57:43.819520 16752 leader_election.cc:304] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:43.821318 16828 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:43.823832 17067 raft_consensus.cc:2743] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:43.836664 17156 raft_consensus.cc:513] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.844530 16830 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:43.845141 16830 ts_tablet_manager.cc:1397] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.127s user 0.010s sys 0.004s
I20250112 20:57:43.848434 16830 raft_consensus.cc:357] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.855237 16830 raft_consensus.cc:383] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.855574 16830 raft_consensus.cc:738] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.858667 16830 consensus_queue.cc:260] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.861443 16830 ts_tablet_manager.cc:1428] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.016s user 0.005s sys 0.002s
I20250112 20:57:43.862674 16828 tablet_bootstrap.cc:654] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.865622 16829 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:43.866160 16829 ts_tablet_manager.cc:1397] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.202s user 0.067s sys 0.108s
I20250112 20:57:43.869532 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2820c3d5974dfcb2230e18411e6493" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:43.869649 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2820c3d5974dfcb2230e18411e6493" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:43.870621 16634 raft_consensus.cc:3054] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.871474 16892 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:43.872166 16894 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:43.872673 16680 leader_election.cc:343] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.870332 17156 leader_election.cc:290] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.878644 16830 tablet_bootstrap.cc:492] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:43.869035 16829 raft_consensus.cc:357] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.880409 16829 raft_consensus.cc:383] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:43.880790 16829 raft_consensus.cc:738] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:43.881417 16829 consensus_queue.cc:260] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.882913 16925 raft_consensus.cc:2743] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:43.883879 16829 ts_tablet_manager.cc:1428] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20250112 20:57:43.885329 16829 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:43.885083 16534 catalog_manager.cc:5526] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:43.899582 16634 raft_consensus.cc:2463] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:43.902072 17173 raft_consensus.cc:491] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.902561 17173 raft_consensus.cc:513] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.904489 17173 leader_election.cc:290] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.905730 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:43.914294 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:43.915138 16707 raft_consensus.cc:2388] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 80770068c3a34f918765f54dbe3a94eb in current term 1: Already voted for candidate c29b243acf2b4f248606563fc96efc7d in this term.
I20250112 20:57:43.912403 17173 raft_consensus.cc:491] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:43.916457 17173 raft_consensus.cc:513] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.919440 16602 leader_election.cc:304] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:43.924762 16678 leader_election.cc:304] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:43.926061 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abeaacbf8c794ec89adddeb5f906f096" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:43.927381 17156 raft_consensus.cc:2798] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:43.927947 17156 raft_consensus.cc:695] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
W20250112 20:57:43.929374 16602 leader_election.cc:343] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.929405 17173 leader_election.cc:290] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:43.930377 16637 raft_consensus.cc:3054] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:43.931264 16830 tablet_bootstrap.cc:654] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:43.934346 17156 consensus_queue.cc:237] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:43.941510 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abeaacbf8c794ec89adddeb5f906f096" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:43.946002 16605 leader_election.cc:343] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:43.946730 16605 leader_election.cc:304] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:43.955093 16891 raft_consensus.cc:2743] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:43.962488 17067 raft_consensus.cc:491] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.055589 16828 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:44.056053 16828 ts_tablet_manager.cc:1397] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.235s user 0.016s sys 0.012s
W20250112 20:57:44.060374 17203 log.cc:926] Time spent T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d: Append to log took a long time: real 0.079s user 0.002s sys 0.004s
I20250112 20:57:44.060871 16829 tablet_bootstrap.cc:654] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.061548 17069 raft_consensus.cc:2743] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:44.062297 16752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:41463, user_credentials={real_user=slave}} blocked reactor thread for 97718.5us
I20250112 20:57:44.062785 17067 raft_consensus.cc:513] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.070230 17067 leader_election.cc:290] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:44.075917 17067 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:44.077450 16830 tablet_bootstrap.cc:492] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:44.077891 16830 ts_tablet_manager.cc:1397] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.199s user 0.016s sys 0.012s
I20250112 20:57:44.079038 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2477026d8d4fac945714c894cca485" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:44.079665 16634 raft_consensus.cc:2463] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:44.081243 16752 leader_election.cc:304] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:44.081975 16875 raft_consensus.cc:2798] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:44.082286 16875 raft_consensus.cc:491] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:44.082540 16875 raft_consensus.cc:3054] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.089947 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2477026d8d4fac945714c894cca485" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:44.090611 16709 raft_consensus.cc:2463] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:44.108121 16828 raft_consensus.cc:357] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.108881 16828 raft_consensus.cc:383] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.109560 16828 raft_consensus.cc:738] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.111711 16828 consensus_queue.cc:260] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.124040 16875 raft_consensus.cc:513] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.132025 16830 raft_consensus.cc:357] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.133042 16830 raft_consensus.cc:383] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.133539 16830 raft_consensus.cc:738] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.134984 16830 consensus_queue.cc:260] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.137296 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2477026d8d4fac945714c894cca485" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:44.138137 16637 raft_consensus.cc:3054] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.137316 16830 ts_tablet_manager.cc:1428] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.059s user 0.007s sys 0.000s
I20250112 20:57:44.140643 16875 leader_election.cc:290] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:44.138923 16535 catalog_manager.cc:5526] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:44.145318 16828 ts_tablet_manager.cc:1428] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.089s user 0.005s sys 0.002s
I20250112 20:57:44.146065 17067 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20250112 20:57:44.150040 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2477026d8d4fac945714c894cca485" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:44.150668 16709 raft_consensus.cc:3054] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.161260 16830 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:44.159571 16637 raft_consensus.cc:2463] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:44.167023 16752 leader_election.cc:304] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:44.167747 16875 raft_consensus.cc:2798] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:44.168148 16875 raft_consensus.cc:695] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:44.168843 16875 consensus_queue.cc:237] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.184043 16828 tablet_bootstrap.cc:492] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:44.198010 16830 tablet_bootstrap.cc:654] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:44.202482 16605 consensus_peers.cc:487] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:44.214356 16829 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:44.214859 16829 ts_tablet_manager.cc:1397] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.330s user 0.018s sys 0.018s
I20250112 20:57:44.215386 16828 tablet_bootstrap.cc:654] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.211390 16709 raft_consensus.cc:2463] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:44.218328 16829 raft_consensus.cc:357] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.219069 16829 raft_consensus.cc:383] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.219347 16829 raft_consensus.cc:738] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.219903 16829 consensus_queue.cc:260] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.222050 16829 ts_tablet_manager.cc:1428] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250112 20:57:44.264796 16830 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:44.265271 16830 ts_tablet_manager.cc:1397] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.120s user 0.012s sys 0.006s
I20250112 20:57:44.268569 16830 raft_consensus.cc:357] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.269398 16830 raft_consensus.cc:383] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.269706 16830 raft_consensus.cc:738] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.270445 16829 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:44.270316 16830 consensus_queue.cc:260] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.272784 16830 ts_tablet_manager.cc:1428] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.007s user 0.001s sys 0.006s
I20250112 20:57:44.273828 16830 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:44.278468 16993 consensus_queue.cc:1035] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250112 20:57:44.283643 16828 tablet_bootstrap.cc:492] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:44.284068 16828 ts_tablet_manager.cc:1397] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.106s user 0.011s sys 0.018s
W20250112 20:57:44.289846 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:44.292867 16537 catalog_manager.cc:5526] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:44.294406 16828 raft_consensus.cc:357] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.295230 16828 raft_consensus.cc:383] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.295482 16828 raft_consensus.cc:738] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.296036 16828 consensus_queue.cc:260] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.299312 16828 ts_tablet_manager.cc:1428] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.015s user 0.003s sys 0.003s
I20250112 20:57:44.304822 16828 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:44.306317 16829 tablet_bootstrap.cc:654] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.313549 16830 tablet_bootstrap.cc:654] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.318632 16637 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.338165 16828 tablet_bootstrap.cc:654] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.342417 16892 consensus_queue.cc:1035] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:44.343456 16829 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:44.343966 17069 raft_consensus.cc:491] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.344412 17069 raft_consensus.cc:513] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.346257 17069 leader_election.cc:290] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:44.347741 16829 ts_tablet_manager.cc:1397] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.102s user 0.013s sys 0.012s
I20250112 20:57:44.347745 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:44.347927 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:44.348533 16711 raft_consensus.cc:2463] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
W20250112 20:57:44.350587 16605 leader_election.cc:343] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.351135 16602 leader_election.cc:304] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:44.352039 17069 raft_consensus.cc:2798] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:44.352365 17069 raft_consensus.cc:491] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:44.352692 17069 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.360942 16829 raft_consensus.cc:357] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.361727 16829 raft_consensus.cc:383] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.361995 16829 raft_consensus.cc:738] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.362644 16829 consensus_queue.cc:260] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.394445 16829 ts_tablet_manager.cc:1428] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.046s user 0.005s sys 0.001s
I20250112 20:57:44.395674 16829 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:44.397513 17069 raft_consensus.cc:513] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.400014 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:44.401335 16711 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.414476 16829 tablet_bootstrap.cc:654] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.416255 16828 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:44.416728 16828 ts_tablet_manager.cc:1397] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.113s user 0.012s sys 0.011s
I20250112 20:57:44.415305 16711 raft_consensus.cc:2463] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:44.418715 16602 leader_election.cc:304] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:44.421406 17069 leader_election.cc:290] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:44.422291 16680 consensus_peers.cc:487] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:44.422324 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:44.423791 17173 raft_consensus.cc:2798] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:44.424269 17173 raft_consensus.cc:695] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
W20250112 20:57:44.424299 16605 leader_election.cc:343] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.425190 17173 consensus_queue.cc:237] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.429631 16828 raft_consensus.cc:357] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.430259 16828 raft_consensus.cc:383] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.430496 16828 raft_consensus.cc:738] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.440701 16830 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:44.441211 16830 ts_tablet_manager.cc:1397] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.168s user 0.017s sys 0.016s
I20250112 20:57:44.442284 16828 consensus_queue.cc:260] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.443796 16830 raft_consensus.cc:357] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.444515 16830 raft_consensus.cc:383] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.444855 16830 raft_consensus.cc:738] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.445530 16830 consensus_queue.cc:260] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.447237 16828 ts_tablet_manager.cc:1428] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20250112 20:57:44.448035 16830 ts_tablet_manager.cc:1428] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:57:44.448197 16828 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:44.450069 16830 tablet_bootstrap.cc:492] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:44.456205 16537 catalog_manager.cc:5526] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:44.464012 16829 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:44.464440 16829 ts_tablet_manager.cc:1397] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.069s user 0.010s sys 0.000s
I20250112 20:57:44.466836 16829 raft_consensus.cc:357] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.467495 16829 raft_consensus.cc:383] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.467748 16829 raft_consensus.cc:738] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.469214 16829 consensus_queue.cc:260] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.473601 16828 tablet_bootstrap.cc:654] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.478149 16829 ts_tablet_manager.cc:1428] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20250112 20:57:44.479043 16829 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:44.476182 16830 tablet_bootstrap.cc:654] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.484587 16829 tablet_bootstrap.cc:654] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.500938 16830 tablet_bootstrap.cc:492] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:44.502590 16830 ts_tablet_manager.cc:1397] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.013s
I20250112 20:57:44.506985 16830 raft_consensus.cc:357] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.504293 16828 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:44.508037 16828 ts_tablet_manager.cc:1397] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.060s user 0.013s sys 0.001s
I20250112 20:57:44.508484 16829 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:44.509636 16829 ts_tablet_manager.cc:1397] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.001s
I20250112 20:57:44.511179 16830 raft_consensus.cc:383] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.511535 16830 raft_consensus.cc:738] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.511657 16829 raft_consensus.cc:357] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.512645 16830 consensus_queue.cc:260] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.512388 16828 raft_consensus.cc:357] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.514596 16828 raft_consensus.cc:383] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.515170 16828 raft_consensus.cc:738] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.515712 16828 consensus_queue.cc:260] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.513129 16829 raft_consensus.cc:383] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.516407 16830 ts_tablet_manager.cc:1428] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.012s user 0.005s sys 0.002s
I20250112 20:57:44.516541 16829 raft_consensus.cc:738] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.517700 16830 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:44.519471 16828 ts_tablet_manager.cc:1428] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.010s user 0.003s sys 0.002s
I20250112 20:57:44.517360 16829 consensus_queue.cc:260] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.520799 16828 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:44.526109 16829 ts_tablet_manager.cc:1428] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20250112 20:57:44.526324 16828 tablet_bootstrap.cc:654] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.529192 17069 raft_consensus.cc:491] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.534629 16829 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:44.534215 17069 raft_consensus.cc:513] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.536280 17069 leader_election.cc:290] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:44.537221 17098 raft_consensus.cc:491] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.537636 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1773dc29c9e4d83a01c6b56f7b277d5" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:44.538159 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1773dc29c9e4d83a01c6b56f7b277d5" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:44.538829 16782 raft_consensus.cc:2463] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:44.537655 17098 raft_consensus.cc:513] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.539877 16605 leader_election.cc:304] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:44.540405 17098 leader_election.cc:290] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:44.540586 17173 raft_consensus.cc:2798] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:44.540953 17173 raft_consensus.cc:491] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:44.541283 17173 raft_consensus.cc:3054] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.538269 16711 raft_consensus.cc:2463] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:44.544770 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abeaacbf8c794ec89adddeb5f906f096" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:44.544215 17208 raft_consensus.cc:491] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.544891 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abeaacbf8c794ec89adddeb5f906f096" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:44.545742 16637 raft_consensus.cc:2463] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:44.545449 17208 raft_consensus.cc:513] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:44.546934 16680 leader_election.cc:343] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.548260 16678 leader_election.cc:304] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:44.550879 17098 raft_consensus.cc:2798] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:44.553253 16894 raft_consensus.cc:491] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.553428 17098 raft_consensus.cc:491] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:44.553938 17098 raft_consensus.cc:3054] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.553731 16894 raft_consensus.cc:513] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.558584 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f61fd1d8bf44f64b73ade11c0224df0" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:44.602326 16829 tablet_bootstrap.cc:654] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
W20250112 20:57:44.574344 16678 leader_election.cc:343] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.615851 16875 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:44.617167 16830 tablet_bootstrap.cc:654] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.575686 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:44.620704 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:44.556489 16894 leader_election.cc:290] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:44.625187 16752 leader_election.cc:343] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.559273 17208 leader_election.cc:290] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:44.613157 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f61fd1d8bf44f64b73ade11c0224df0" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
W20250112 20:57:44.627784 16751 leader_election.cc:343] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.628161 16751 leader_election.cc:304] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:44.629129 17208 raft_consensus.cc:2743] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:44.637794 16785 raft_consensus.cc:3054] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.639885 17098 raft_consensus.cc:513] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:44.636936 16680 leader_election.cc:343] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.642661 16680 leader_election.cc:304] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:44.643435 17156 raft_consensus.cc:2743] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:44.647707 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abeaacbf8c794ec89adddeb5f906f096" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:44.648707 17098 leader_election.cc:290] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:44.649920 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abeaacbf8c794ec89adddeb5f906f096" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:44.650486 16638 raft_consensus.cc:3054] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.650715 17173 raft_consensus.cc:513] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.657486 16638 raft_consensus.cc:2463] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:44.657810 17173 leader_election.cc:290] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:44.661388 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1773dc29c9e4d83a01c6b56f7b277d5" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:44.662019 16711 raft_consensus.cc:3054] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.668377 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1773dc29c9e4d83a01c6b56f7b277d5" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:44.669041 16782 raft_consensus.cc:3054] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.670686 16828 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:44.671514 16828 ts_tablet_manager.cc:1397] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.151s user 0.013s sys 0.008s
W20250112 20:57:44.675931 16680 leader_election.cc:343] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.676210 16711 raft_consensus.cc:2463] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:44.687407 16602 leader_election.cc:304] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:44.688318 16891 raft_consensus.cc:491] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.688850 16678 leader_election.cc:304] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:44.691308 17172 raft_consensus.cc:2798] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:44.691510 17069 raft_consensus.cc:2798] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:44.691800 17172 raft_consensus.cc:695] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:44.691949 17069 raft_consensus.cc:695] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:44.692603 17172 consensus_queue.cc:237] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.688863 16891 raft_consensus.cc:513] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.692750 17069 consensus_queue.cc:237] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.694274 16828 raft_consensus.cc:357] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.698491 16828 raft_consensus.cc:383] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.698885 16828 raft_consensus.cc:738] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.702987 16875 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:44.710922 16828 consensus_queue.cc:260] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.714834 16891 leader_election.cc:290] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:44.733040 16829 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:44.733515 16829 ts_tablet_manager.cc:1397] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.199s user 0.013s sys 0.012s
I20250112 20:57:44.745296 16828 ts_tablet_manager.cc:1428] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.072s user 0.007s sys 0.001s
I20250112 20:57:44.747110 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:44.748121 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:44.752205 16782 raft_consensus.cc:2463] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
W20250112 20:57:44.752452 16602 leader_election.cc:343] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.754767 17172 raft_consensus.cc:491] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.755213 17172 raft_consensus.cc:513] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.758119 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:44.758931 17172 leader_election.cc:290] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:44.759711 16680 leader_election.cc:343] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.763280 16829 raft_consensus.cc:357] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.764878 16829 raft_consensus.cc:383] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.766093 16829 raft_consensus.cc:738] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.765974 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:44.766690 16829 consensus_queue.cc:260] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.768107 16828 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
W20250112 20:57:44.768777 16678 leader_election.cc:343] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.769224 16678 leader_election.cc:304] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 018d03f043b542b9b63491a77523968d; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:44.770155 17156 raft_consensus.cc:2743] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:44.779466 16830 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:44.779919 16830 ts_tablet_manager.cc:1397] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.262s user 0.025s sys 0.027s
I20250112 20:57:44.798641 16537 catalog_manager.cc:5526] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:44.803475 16875 raft_consensus.cc:491] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.803956 16875 raft_consensus.cc:513] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.812320 16875 leader_election.cc:290] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:44.817755 16829 ts_tablet_manager.cc:1428] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.084s user 0.007s sys 0.004s
I20250112 20:57:44.818764 16829 tablet_bootstrap.cc:492] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:44.822425 16828 tablet_bootstrap.cc:654] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.832114 16891 raft_consensus.cc:491] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:44.832573 16891 raft_consensus.cc:513] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.834003 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:44.834267 16891 leader_election.cc:290] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:57:44.837741 16605 leader_election.cc:343] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.839196 16605 leader_election.cc:304] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:44.840114 16891 raft_consensus.cc:2743] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:57:44.844070 16751 leader_election.cc:343] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.844137 16635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:44.860913 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdb0f016b8946c582e26ab6e02cb9b1" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:44.862170 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdb0f016b8946c582e26ab6e02cb9b1" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
W20250112 20:57:44.864656 16602 leader_election.cc:343] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:57:44.866531 16752 leader_election.cc:343] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.870049 16752 leader_election.cc:304] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:44.870579 17237 raft_consensus.cc:2743] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:44.875789 16830 raft_consensus.cc:357] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.876648 16830 raft_consensus.cc:383] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.876947 16830 raft_consensus.cc:738] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
W20250112 20:57:44.873780 16605 leader_election.cc:343] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:44.893895 16605 leader_election.cc:304] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:44.894745 16830 consensus_queue.cc:260] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.895026 17234 raft_consensus.cc:2743] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:44.901070 16891 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:44.901866 16534 catalog_manager.cc:5526] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:44.904628 16829 tablet_bootstrap.cc:654] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.907986 16828 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:44.908468 16828 ts_tablet_manager.cc:1397] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.157s user 0.022s sys 0.026s
I20250112 20:57:44.910190 16783 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:44.910794 16828 raft_consensus.cc:357] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.911424 16828 raft_consensus.cc:383] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.911682 16828 raft_consensus.cc:738] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.912688 16828 consensus_queue.cc:260] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.914717 16828 ts_tablet_manager.cc:1428] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:57:44.917719 16828 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:44.920238 17013 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250112 20:57:44.925704 16828 tablet_bootstrap.cc:654] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.934520 16830 ts_tablet_manager.cc:1428] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.154s user 0.006s sys 0.010s
I20250112 20:57:44.935519 16830 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:44.958174 16828 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:44.958779 16828 ts_tablet_manager.cc:1397] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.020s
I20250112 20:57:44.960441 16829 tablet_bootstrap.cc:492] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:44.961946 16829 ts_tablet_manager.cc:1397] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.143s user 0.007s sys 0.005s
I20250112 20:57:44.964831 16828 raft_consensus.cc:357] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.965507 16828 raft_consensus.cc:383] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.965751 16828 raft_consensus.cc:738] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.966274 16828 consensus_queue.cc:260] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.968286 16828 ts_tablet_manager.cc:1428] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20250112 20:57:44.971345 16829 raft_consensus.cc:357] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.976416 16829 raft_consensus.cc:383] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:44.976902 16829 raft_consensus.cc:738] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:44.977577 16829 consensus_queue.cc:260] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:44.990051 16828 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:44.982899 16830 tablet_bootstrap.cc:654] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:44.993359 16829 ts_tablet_manager.cc:1428] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.031s user 0.002s sys 0.003s
I20250112 20:57:45.001484 16829 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:45.016292 16829 tablet_bootstrap.cc:654] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.005904 16828 tablet_bootstrap.cc:654] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.018245 17013 raft_consensus.cc:491] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.030545 17013 raft_consensus.cc:513] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.032509 17013 leader_election.cc:290] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.023118 16830 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:45.033931 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:45.034574 16707 raft_consensus.cc:2463] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:45.036108 16602 leader_election.cc:304] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:45.037070 16891 raft_consensus.cc:2798] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:45.037405 16891 raft_consensus.cc:491] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:45.037732 16891 raft_consensus.cc:3054] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.039417 16830 ts_tablet_manager.cc:1397] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.104s user 0.005s sys 0.022s
I20250112 20:57:45.040546 16829 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:45.041386 16829 ts_tablet_manager.cc:1397] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.040s user 0.006s sys 0.007s
I20250112 20:57:45.042886 16830 raft_consensus.cc:357] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.043615 16830 raft_consensus.cc:383] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.041693 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:45.042806 16828 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:45.044435 16828 ts_tablet_manager.cc:1397] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.062s user 0.011s sys 0.001s
W20250112 20:57:45.045389 16605 leader_election.cc:343] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:45.047004 16828 raft_consensus.cc:357] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.047773 16828 raft_consensus.cc:383] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.043860 16830 raft_consensus.cc:738] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.050112 16828 raft_consensus.cc:738] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.050498 16830 consensus_queue.cc:260] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.050846 16828 consensus_queue.cc:260] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.052791 16828 ts_tablet_manager.cc:1428] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20250112 20:57:45.053649 16828 tablet_bootstrap.cc:492] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d: Bootstrap starting.
I20250112 20:57:45.059026 16828 tablet_bootstrap.cc:654] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.049443 16829 raft_consensus.cc:357] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.064824 16829 raft_consensus.cc:383] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.065089 16829 raft_consensus.cc:738] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.065696 16829 consensus_queue.cc:260] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.067983 16829 ts_tablet_manager.cc:1428] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.025s user 0.004s sys 0.002s
I20250112 20:57:45.069842 16829 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:45.063055 16891 raft_consensus.cc:513] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.060530 16830 ts_tablet_manager.cc:1428] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.021s user 0.005s sys 0.001s
I20250112 20:57:45.073750 16830 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:45.080482 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:45.080713 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:45.081107 16707 raft_consensus.cc:3054] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:57:45.081946 16605 leader_election.cc:343] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:45.085069 16891 leader_election.cc:290] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.088248 16707 raft_consensus.cc:2463] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:45.089388 16602 leader_election.cc:304] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:45.089754 16829 tablet_bootstrap.cc:654] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.097967 16830 tablet_bootstrap.cc:654] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.100452 17013 raft_consensus.cc:2798] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:45.100935 17013 raft_consensus.cc:695] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:45.101676 17013 consensus_queue.cc:237] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.114629 16828 tablet_bootstrap.cc:492] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d: No bootstrap required, opened a new log
I20250112 20:57:45.115096 16828 ts_tablet_manager.cc:1397] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d: Time spent bootstrapping tablet: real 0.062s user 0.018s sys 0.010s
I20250112 20:57:45.117753 16828 raft_consensus.cc:357] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.118467 16828 raft_consensus.cc:383] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.118737 16828 raft_consensus.cc:738] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.119339 16828 consensus_queue.cc:260] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.121609 16828 ts_tablet_manager.cc:1428] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:57:45.123486 16830 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:45.123924 16830 ts_tablet_manager.cc:1397] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.050s user 0.010s sys 0.003s
I20250112 20:57:45.124755 16535 catalog_manager.cc:5526] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:45.126227 16830 raft_consensus.cc:357] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.126765 16830 raft_consensus.cc:383] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.126989 16830 raft_consensus.cc:738] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.129397 16829 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:45.130043 16829 ts_tablet_manager.cc:1397] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.060s user 0.011s sys 0.003s
I20250112 20:57:45.132719 16829 raft_consensus.cc:357] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.133319 16829 raft_consensus.cc:383] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.133564 16829 raft_consensus.cc:738] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.134124 16829 consensus_queue.cc:260] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.136269 16829 ts_tablet_manager.cc:1428] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20250112 20:57:45.137570 16830 consensus_queue.cc:260] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.139707 16830 ts_tablet_manager.cc:1428] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20250112 20:57:45.140560 16830 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb: Bootstrap starting.
I20250112 20:57:45.142603 17069 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:45.143517 16829 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:45.149545 17013 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:45.162804 16830 tablet_bootstrap.cc:654] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.166361 16829 tablet_bootstrap.cc:654] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.183055 16829 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:45.183509 16829 ts_tablet_manager.cc:1397] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.040s user 0.013s sys 0.002s
I20250112 20:57:45.183269 17098 consensus_queue.cc:1035] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:45.185981 16829 raft_consensus.cc:357] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.186638 16829 raft_consensus.cc:383] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.186942 16829 raft_consensus.cc:738] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.187680 16829 consensus_queue.cc:260] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.209075 16829 ts_tablet_manager.cc:1428] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.025s user 0.006s sys 0.002s
I20250112 20:57:45.219305 16785 raft_consensus.cc:3054] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.210660 16830 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb: No bootstrap required, opened a new log
I20250112 20:57:45.224231 16830 ts_tablet_manager.cc:1397] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb: Time spent bootstrapping tablet: real 0.084s user 0.014s sys 0.002s
I20250112 20:57:45.224396 16829 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:45.227069 16830 raft_consensus.cc:357] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.228006 16830 raft_consensus.cc:383] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.228315 16830 raft_consensus.cc:738] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.242364 16830 consensus_queue.cc:260] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.249506 17098 raft_consensus.cc:491] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.250882 16830 ts_tablet_manager.cc:1428] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb: Time spent starting tablet: real 0.026s user 0.006s sys 0.001s
I20250112 20:57:45.252786 17172 raft_consensus.cc:491] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.253249 17172 raft_consensus.cc:513] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.253638 17236 raft_consensus.cc:491] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.254106 17236 raft_consensus.cc:513] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.255121 17172 leader_election.cc:290] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.255920 17236 leader_election.cc:290] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:45.257416 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdb0f016b8946c582e26ab6e02cb9b1" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:45.258051 16783 raft_consensus.cc:2463] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:45.249985 17098 raft_consensus.cc:513] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.262162 17098 leader_election.cc:290] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.262890 16680 leader_election.cc:304] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:45.263448 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f61fd1d8bf44f64b73ade11c0224df0" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:45.264169 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:45.265944 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f61fd1d8bf44f64b73ade11c0224df0" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:45.266597 16712 raft_consensus.cc:2463] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:45.266681 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdb0f016b8946c582e26ab6e02cb9b1" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:45.267289 16634 raft_consensus.cc:2463] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:45.264091 16637 raft_consensus.cc:2463] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 0.
I20250112 20:57:45.268673 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:45.269236 16634 raft_consensus.cc:2463] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:45.272177 17156 raft_consensus.cc:2798] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:45.272558 17156 raft_consensus.cc:491] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:45.272715 16751 leader_election.cc:304] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:45.272923 17156 raft_consensus.cc:3054] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.273662 17237 raft_consensus.cc:2798] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:45.274016 17237 raft_consensus.cc:491] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:45.274339 17237 raft_consensus.cc:3054] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.277415 17156 raft_consensus.cc:513] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:45.278849 16680 leader_election.cc:343] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:45.280645 17237 raft_consensus.cc:513] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.282814 17156 leader_election.cc:290] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.284258 16678 leader_election.cc:304] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters: c29b243acf2b4f248606563fc96efc7d
I20250112 20:57:45.284538 17237 leader_election.cc:290] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:45.286450 16894 raft_consensus.cc:2798] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:45.286804 16894 raft_consensus.cc:491] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:45.287118 16894 raft_consensus.cc:3054] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.287863 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f61fd1d8bf44f64b73ade11c0224df0" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:45.288467 16638 raft_consensus.cc:3054] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.288821 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f61fd1d8bf44f64b73ade11c0224df0" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:45.289467 16707 raft_consensus.cc:3054] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.289559 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdb0f016b8946c582e26ab6e02cb9b1" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:45.290189 16782 raft_consensus.cc:3054] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.295051 16638 raft_consensus.cc:2463] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:45.299993 16707 raft_consensus.cc:2463] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 1.
I20250112 20:57:45.300938 16751 leader_election.cc:304] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:45.301810 17237 raft_consensus.cc:2798] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:45.302560 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bdb0f016b8946c582e26ab6e02cb9b1" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:45.302907 17237 raft_consensus.cc:695] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:57:45.303130 17156 consensus_queue.cc:1035] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:45.303925 17237 consensus_queue.cc:237] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.307154 16637 raft_consensus.cc:3054] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.320830 17098 raft_consensus.cc:491] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.322280 17098 raft_consensus.cc:513] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.324929 16894 raft_consensus.cc:513] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.326800 16782 raft_consensus.cc:2463] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:45.327713 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:45.328001 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:45.328248 17098 leader_election.cc:290] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.327806 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:45.328393 16638 raft_consensus.cc:3054] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.328662 16634 raft_consensus.cc:2463] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:45.329584 16678 leader_election.cc:304] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:45.331790 16783 raft_consensus.cc:2463] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 0.
I20250112 20:57:45.335120 17098 raft_consensus.cc:2798] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:45.334693 16638 raft_consensus.cc:2463] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:45.336117 17098 raft_consensus.cc:491] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:45.337174 16637 raft_consensus.cc:2463] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:45.338032 16894 leader_election.cc:290] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.338462 16680 leader_election.cc:304] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:45.338980 16894 raft_consensus.cc:2798] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:45.336467 17098 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.339409 16678 leader_election.cc:304] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:45.341121 17156 raft_consensus.cc:2798] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:45.341552 17156 raft_consensus.cc:695] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:45.342336 17156 consensus_queue.cc:237] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.343904 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:45.339007 17234 raft_consensus.cc:491] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.344935 16894 raft_consensus.cc:695] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:45.335022 16829 tablet_bootstrap.cc:654] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.346320 17234 raft_consensus.cc:513] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.346735 16894 consensus_queue.cc:237] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:45.349079 16680 leader_election.cc:343] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:57:45.352670 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:45.353240 16707 raft_consensus.cc:2463] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:45.354285 16602 leader_election.cc:304] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:45.355152 17173 raft_consensus.cc:2798] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:57:45.355489 17237 raft_consensus.cc:491] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.355557 17173 raft_consensus.cc:491] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:57:45.356144 17173 raft_consensus.cc:3054] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.356076 17237 raft_consensus.cc:513] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.367311 16829 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:45.367902 16829 ts_tablet_manager.cc:1397] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.144s user 0.003s sys 0.008s
I20250112 20:57:45.334125 16537 catalog_manager.cc:5526] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 0 to 1, leader changed from <none> to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:57:45.370779 17234 leader_election.cc:290] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.380836 16829 raft_consensus.cc:357] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.381537 16829 raft_consensus.cc:383] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.381764 16829 raft_consensus.cc:738] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.382287 16829 consensus_queue.cc:260] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.383744 17098 raft_consensus.cc:513] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.386245 17098 leader_election.cc:290] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.387887 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:45.388505 16783 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.388952 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:57:45.389484 16637 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.383162 16782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d" is_pre_election: true
I20250112 20:57:45.391299 16782 raft_consensus.cc:2463] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 0.
I20250112 20:57:45.394742 16783 raft_consensus.cc:2463] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:45.396991 16680 leader_election.cc:304] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:57:45.397732 17172 raft_consensus.cc:2798] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:45.398135 17172 raft_consensus.cc:695] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:57:45.398896 17172 consensus_queue.cc:237] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.399842 17173 raft_consensus.cc:513] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
W20250112 20:57:45.405459 16585 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:57:45.407207 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2820c3d5974dfcb2230e18411e6493" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:45.409471 17237 leader_election.cc:290] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:45.408885 16534 catalog_manager.cc:5526] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:45.411942 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:57:45.412454 16707 raft_consensus.cc:3054] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.412904 16638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc2820c3d5974dfcb2230e18411e6493" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:45.413460 17173 leader_election.cc:290] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:45.414283 16752 leader_election.cc:304] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:45.415459 17236 raft_consensus.cc:2743] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:45.425026 16637 raft_consensus.cc:2463] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 1.
I20250112 20:57:45.430917 16829 ts_tablet_manager.cc:1428] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.063s user 0.010s sys 0.002s
I20250112 20:57:45.431946 16829 tablet_bootstrap.cc:492] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d: Bootstrap starting.
I20250112 20:57:45.439921 16829 tablet_bootstrap.cc:654] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d: Neither blocks nor log segments found. Creating new log.
I20250112 20:57:45.445065 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:57:45.445724 16783 raft_consensus.cc:3054] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.431946 16707 raft_consensus.cc:2463] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:45.463727 16829 tablet_bootstrap.cc:492] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d: No bootstrap required, opened a new log
I20250112 20:57:45.465425 16602 leader_election.cc:304] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:57:45.465818 16829 ts_tablet_manager.cc:1397] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.008s
I20250112 20:57:45.466147 17173 raft_consensus.cc:2798] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:57:45.466624 17173 raft_consensus.cc:695] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:57:45.467387 17173 consensus_queue.cc:237] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.474280 16829 raft_consensus.cc:357] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.475356 16829 raft_consensus.cc:383] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:57:45.475646 16829 raft_consensus.cc:738] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Initialized, Role: FOLLOWER
I20250112 20:57:45.479454 16829 consensus_queue.cc:260] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.481775 16783 raft_consensus.cc:2463] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 1.
I20250112 20:57:45.525728 17236 raft_consensus.cc:491] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.526186 17236 raft_consensus.cc:513] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.527236 16783 raft_consensus.cc:3054] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.527989 17236 leader_election.cc:290] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:45.528538 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:45.529807 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:45.530376 16709 raft_consensus.cc:2388] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c29b243acf2b4f248606563fc96efc7d in current term 1: Already voted for candidate 80770068c3a34f918765f54dbe3a94eb in this term.
I20250112 20:57:45.531185 16751 leader_election.cc:304] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:45.531841 17236 raft_consensus.cc:3054] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.535986 16534 catalog_manager.cc:5526] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:45.538069 17236 raft_consensus.cc:2743] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:45.532380 16537 catalog_manager.cc:5526] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:45.540792 16537 catalog_manager.cc:5526] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 0 to 1, leader changed from <none> to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:45.553166 16829 ts_tablet_manager.cc:1428] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d: Time spent starting tablet: real 0.087s user 0.010s sys 0.034s
I20250112 20:57:45.555544 17173 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:45.576936 17013 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:45.646327 17261 raft_consensus.cc:491] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:57:45.646808 17261 raft_consensus.cc:513] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:57:45.648901 17261 leader_election.cc:290] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:45.650123 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80770068c3a34f918765f54dbe3a94eb" is_pre_election: true
I20250112 20:57:45.650815 16637 raft_consensus.cc:2388] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c29b243acf2b4f248606563fc96efc7d in current term 1: Already voted for candidate 018d03f043b542b9b63491a77523968d in this term.
I20250112 20:57:45.650997 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018d03f043b542b9b63491a77523968d" is_pre_election: true
I20250112 20:57:45.654910 16751 leader_election.cc:304] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:57:45.658492 17261 raft_consensus.cc:3054] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:57:45.688252 17261 raft_consensus.cc:2743] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:57:45.727131 17261 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:45.748692 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250112 20:57:45.767263 16894 consensus_queue.cc:1035] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:57:45.772111 17175 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:45.785013 17269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:57:45.788619 17270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:45.823945 16894 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:45.835621 17098 consensus_queue.cc:1035] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250112 20:57:45.843015 17274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:57:45.896525 16891 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:57:45.906666 17276 log.cc:926] Time spent T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d: Append to log took a long time: real 0.056s user 0.001s sys 0.000s
I20250112 20:57:45.916169 15454 server_base.cc:1034] running on GCE node
I20250112 20:57:45.918797 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:57:45.919085 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:57:45.919274 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715465919257 us; error 0 us; skew 500 ppm
I20250112 20:57:45.919956 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:57:45.937058 17098 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:45.943806 17257 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:45.950382 17257 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:46.026036 17173 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:57:46.155478 15454 webserver.cc:458] Webserver started at http://127.15.23.132:44123/ using document root <none> and password file <none>
I20250112 20:57:46.156280 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:57:46.156565 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:57:46.157004 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:57:46.158855 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-3-root/instance:
uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
format_stamp: "Formatted at 2025-01-12 20:57:46 on dist-test-slave-m2nm"
I20250112 20:57:46.193531 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.034s user 0.007s sys 0.005s
I20250112 20:57:46.213371 17286 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:46.214439 15454 fs_manager.cc:730] Time spent opening block manager: real 0.014s user 0.000s sys 0.004s
I20250112 20:57:46.214900 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-3-root
uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
format_stamp: "Formatted at 2025-01-12 20:57:46 on dist-test-slave-m2nm"
I20250112 20:57:46.215394 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:57:46.320902 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:57:46.322590 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:57:46.361094 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:57:46.364446 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:57:46.364756 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250112 20:57:46.365072 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:57:46.365286 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:57:46.678689 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.132:43633
I20250112 20:57:46.679036 17349 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.132:43633 every 8 connection(s)
I20250112 20:57:46.705013 15454 auto_leader_rebalancer-test.cc:413] add a tserver: 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:46.717830 17350 heartbeater.cc:346] Connected to a master server at 127.15.23.190:44397
I20250112 20:57:46.718291 17350 heartbeater.cc:463] Registering TS with master...
I20250112 20:57:46.719126 17350 heartbeater.cc:510] Master 127.15.23.190:44397 requested a full tablet report, sending...
I20250112 20:57:46.721278 16537 ts_manager.cc:194] Registered new tserver with Master: 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633)
I20250112 20:57:46.722966 16537 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52694
I20250112 20:57:47.632875 16782 consensus_queue.cc:237] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:47.651981 16709 raft_consensus.cc:1270] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:47.653699 17261 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:47.677425 16635 raft_consensus.cc:1270] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:47.679524 17261 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:47.685158 17175 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:47.696511 16709 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:47.708153 16782 consensus_queue.cc:237] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:47.715312 16537 catalog_manager.cc:5526] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250112 20:57:47.718107 16752 consensus_peers.cc:487] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e3b5e20d22c4251b1e2b91ca0479e2f. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:47.715391 16634 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:47.730700 17350 heartbeater.cc:502] Master 127.15.23.190:44397 was elected leader, sending a full tablet report...
I20250112 20:57:47.730996 16712 raft_consensus.cc:1270] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:47.735311 17364 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:57:47.755227 16752 consensus_peers.cc:487] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d2477026d8d4fac945714c894cca485. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:47.758692 16637 raft_consensus.cc:1270] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:47.770916 17261 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:47.782990 16709 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:47.814576 17261 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:47.823691 16535 catalog_manager.cc:5526] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:47.857215 16635 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:47.893419 16637 consensus_queue.cc:237] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:47.908613 16783 raft_consensus.cc:1270] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:47.908677 16709 raft_consensus.cc:1270] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:47.909899 17013 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:47.911458 17013 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:57:47.935618 16603 consensus_peers.cc:487] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d979f5162b594f329ba9439c121741de. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:47.942378 17013 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:47.956404 16637 consensus_queue.cc:237] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:47.972112 16782 raft_consensus.cc:1270] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:47.976218 17376 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:47.977054 16535 catalog_manager.cc:5526] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:47.988951 16707 raft_consensus.cc:1270] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:47.990622 16891 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:57:48.000064 16603 consensus_peers.cc:487] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ffbc5c73802406893d78f89cf5c145c. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:48.007449 16785 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:48.041615 16709 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:48.066013 17374 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:48.071293 16709 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:48.090242 16785 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:48.091802 16535 catalog_manager.cc:5526] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:48.218490 17383 ts_tablet_manager.cc:927] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:48.221256 17383 tablet_copy_client.cc:323] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:57:48.278324 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet 3d2477026d8d4fac945714c894cca485 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:57:48.279049 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet 3d2477026d8d4fac945714c894cca485 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-3d2477026d8d4fac945714c894cca485
I20250112 20:57:48.291424 16798 tablet_copy_source_session.cc:215] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:48.307696 17383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d2477026d8d4fac945714c894cca485. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:48.319900 17388 ts_tablet_manager.cc:927] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:48.301491 17387 ts_tablet_manager.cc:927] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:48.336014 17387 tablet_copy_client.cc:323] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:57:48.337911 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:57:48.338479 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet 3e3b5e20d22c4251b1e2b91ca0479e2f from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-3e3b5e20d22c4251b1e2b91ca0479e2f
I20250112 20:57:48.360756 17388 tablet_copy_client.cc:323] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:57:48.362692 16798 tablet_copy_source_session.cc:215] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:48.374410 17383 tablet_copy_client.cc:806] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:48.375676 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet d979f5162b594f329ba9439c121741de from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:57:48.378175 17387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:48.378733 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet d979f5162b594f329ba9439c121741de from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-d979f5162b594f329ba9439c121741de
I20250112 20:57:48.380139 17383 tablet_copy_client.cc:670] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:48.395181 16649 tablet_copy_source_session.cc:215] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:48.396687 17383 tablet_copy_client.cc:538] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:48.398206 17388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d979f5162b594f329ba9439c121741de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:48.432854 17383 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:48.444972 17387 tablet_copy_client.cc:806] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:48.445623 17387 tablet_copy_client.cc:670] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:48.449185 17387 tablet_copy_client.cc:538] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:48.440999 17388 tablet_copy_client.cc:806] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:48.461100 17388 tablet_copy_client.cc:670] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:48.464758 17391 ts_tablet_manager.cc:927] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:48.467828 17388 tablet_copy_client.cc:538] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:48.476060 17391 tablet_copy_client.cc:323] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:57:48.478447 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 5ffbc5c73802406893d78f89cf5c145c from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:57:48.478976 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 5ffbc5c73802406893d78f89cf5c145c from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-5ffbc5c73802406893d78f89cf5c145c
I20250112 20:57:48.486179 17388 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:48.488657 16649 tablet_copy_source_session.cc:215] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:48.489101 17387 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:48.500048 17391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ffbc5c73802406893d78f89cf5c145c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:48.522439 17391 tablet_copy_client.cc:806] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:48.522980 17391 tablet_copy_client.cc:670] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:48.529135 17391 tablet_copy_client.cc:538] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:48.536164 17391 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:48.537503 17383 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:48.557899 17383 tablet_bootstrap.cc:492] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:48.558568 17383 ts_tablet_manager.cc:1397] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.135s user 0.023s sys 0.006s
I20250112 20:57:48.548708 17388 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:48.565160 17388 tablet_bootstrap.cc:492] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:48.567201 17388 ts_tablet_manager.cc:1397] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.081s user 0.017s sys 0.006s
I20250112 20:57:48.572532 17383 raft_consensus.cc:357] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:48.576700 17383 raft_consensus.cc:738] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:48.580937 17383 consensus_queue.cc:260] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:48.583576 17387 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:48.586619 17387 tablet_bootstrap.cc:492] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:48.588650 17387 ts_tablet_manager.cc:1397] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.100s user 0.024s sys 0.001s
I20250112 20:57:48.595407 17388 raft_consensus.cc:357] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:48.596634 17388 raft_consensus.cc:738] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:48.597326 17388 consensus_queue.cc:260] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:48.599874 17323 raft_consensus.cc:1212] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:48.603314 17383 ts_tablet_manager.cc:1428] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.044s user 0.011s sys 0.004s
I20250112 20:57:48.605396 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-3d2477026d8d4fac945714c894cca485 received from {username='slave'} at 127.0.0.1:51714
I20250112 20:57:48.605857 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-3d2477026d8d4fac945714c894cca485 on tablet 3d2477026d8d4fac945714c894cca485 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:48.618976 17387 raft_consensus.cc:357] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:48.625675 17387 raft_consensus.cc:738] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:48.626534 17387 consensus_queue.cc:260] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:48.629370 17388 ts_tablet_manager.cc:1428] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.059s user 0.007s sys 0.000s
I20250112 20:57:48.636307 17387 ts_tablet_manager.cc:1428] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
I20250112 20:57:48.649808 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-d979f5162b594f329ba9439c121741de received from {username='slave'} at 127.0.0.1:36254
I20250112 20:57:48.650286 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-d979f5162b594f329ba9439c121741de on tablet d979f5162b594f329ba9439c121741de with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:48.652182 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-3e3b5e20d22c4251b1e2b91ca0479e2f received from {username='slave'} at 127.0.0.1:51714
I20250112 20:57:48.653302 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-3e3b5e20d22c4251b1e2b91ca0479e2f on tablet 3e3b5e20d22c4251b1e2b91ca0479e2f with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:48.729943 17323 raft_consensus.cc:1212] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:48.733840 17391 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:48.734834 17391 tablet_bootstrap.cc:492] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:48.735517 17391 ts_tablet_manager.cc:1397] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.200s user 0.054s sys 0.079s
I20250112 20:57:48.738207 17391 raft_consensus.cc:357] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:48.739109 17391 raft_consensus.cc:738] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:48.739765 17391 consensus_queue.cc:260] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:48.743994 17391 ts_tablet_manager.cc:1428] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.008s user 0.004s sys 0.001s
I20250112 20:57:48.746254 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-5ffbc5c73802406893d78f89cf5c145c received from {username='slave'} at 127.0.0.1:36254
I20250112 20:57:48.746683 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-5ffbc5c73802406893d78f89cf5c145c on tablet 5ffbc5c73802406893d78f89cf5c145c with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:48.897099 17323 raft_consensus.cc:1212] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:49.051970 17323 raft_consensus.cc:1212] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:49.143456 17261 raft_consensus.cc:1059] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:49.145745 17261 consensus_queue.cc:237] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:49.155591 16709 raft_consensus.cc:1270] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.158362 17364 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.167228 17323 raft_consensus.cc:1270] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.173050 16637 raft_consensus.cc:1270] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.179446 17175 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.181274 17175 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.202716 17175 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.215068 16709 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.231743 17323 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.264706 16534 catalog_manager.cc:5526] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:49.268746 17175 raft_consensus.cc:1059] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:49.270895 17175 consensus_queue.cc:237] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:49.276589 17013 raft_consensus.cc:1059] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:49.278646 17013 consensus_queue.cc:237] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:49.298202 17323 raft_consensus.cc:1270] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.299619 16709 raft_consensus.cc:1270] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.302608 16638 raft_consensus.cc:1270] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.303511 17261 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.306455 17364 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.310246 17261 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.324420 16638 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.343798 17403 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.348616 16709 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.369316 16638 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.390969 16714 raft_consensus.cc:1270] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.393975 17411 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.395493 17323 raft_consensus.cc:1270] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.399353 16783 raft_consensus.cc:1270] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.400247 17376 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.407692 17411 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.407125 17323 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
W20250112 20:57:49.436990 17407 log.cc:926] Time spent T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: Append to log took a long time: real 0.067s user 0.000s sys 0.001s
I20250112 20:57:49.467955 17013 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.479766 16788 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.495901 17323 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.512208 16714 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.599341 17417 raft_consensus.cc:1059] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:49.601804 17417 consensus_queue.cc:237] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:49.615168 16534 catalog_manager.cc:5526] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.622148 17323 raft_consensus.cc:1270] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.623543 17417 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.623948 16534 catalog_manager.cc:5526] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:57:49.630960 16788 raft_consensus.cc:1270] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.633642 17417 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.634542 16714 raft_consensus.cc:1270] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:49.648772 17417 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:49.677594 16635 consensus_queue.cc:237] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:49.683658 17324 raft_consensus.cc:1270] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:49.685009 17374 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:49.690694 16784 consensus_queue.cc:237] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:49.693274 16709 raft_consensus.cc:1270] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:49.696422 16783 consensus_queue.cc:237] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:49.696249 17374 consensus_queue.cc:1035] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:49.700896 17013 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.703450 16709 raft_consensus.cc:1270] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:49.702776 17324 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.707168 17322 raft_consensus.cc:1270] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:49.712788 17404 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20250112 20:57:49.715427 17175 consensus_queue.cc:1035] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:49.716023 16638 raft_consensus.cc:1270] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:49.717135 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet d979f5162b594f329ba9439c121741de with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:49.717936 17403 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:49.721156 17322 raft_consensus.cc:1270] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:49.722090 16709 raft_consensus.cc:2949] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.723203 17403 consensus_queue.cc:1035] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:49.721587 16534 catalog_manager.cc:5526] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:57:49.741322 17013 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.748842 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet d979f5162b594f329ba9439c121741de with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:57:49.760371 17175 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.762717 16709 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.762969 16638 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.757202 17364 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.757668 16707 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.771308 16787 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.772385 17324 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.784855 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 3d2477026d8d4fac945714c894cca485 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:49.785265 17323 raft_consensus.cc:2949] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.781972 17322 raft_consensus.cc:2949] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.796492 16537 catalog_manager.cc:5526] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
W20250112 20:57:49.860224 16751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:38087, user_credentials={real_user=slave}} blocked reactor thread for 50948.4us
I20250112 20:57:49.865455 16532 catalog_manager.cc:5526] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.869582 17423 tablet_replica.cc:331] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:57:49.879634 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d2477026d8d4fac945714c894cca485 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:57:49.884444 17423 raft_consensus.cc:2238] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:49.885221 17423 raft_consensus.cc:2267] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:49.894678 16532 catalog_manager.cc:5526] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.898767 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:49.902294 17423 ts_tablet_manager.cc:1905] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:49.908880 17426 tablet_replica.cc:331] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:57:49.910445 17426 raft_consensus.cc:2238] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:49.917847 17426 raft_consensus.cc:2267] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:49.929520 17426 ts_tablet_manager.cc:1905] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:49.933573 17423 ts_tablet_manager.cc:1918] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:49.934113 17423 log.cc:1198] T d979f5162b594f329ba9439c121741de P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/d979f5162b594f329ba9439c121741de
I20250112 20:57:49.935951 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet d979f5162b594f329ba9439c121741de (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:49.944509 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3b5e20d22c4251b1e2b91ca0479e2f with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:57:49.952236 16637 consensus_queue.cc:237] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:49.957743 17324 raft_consensus.cc:1270] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:49.959697 16788 raft_consensus.cc:1270] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:49.960994 17013 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:49.962026 17376 consensus_queue.cc:1035] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:49.981768 17417 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.994422 17324 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:49.998804 16787 raft_consensus.cc:2949] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:50.017087 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 5ffbc5c73802406893d78f89cf5c145c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:50.011101 17426 ts_tablet_manager.cc:1918] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:50.019783 17426 log.cc:1198] T 3d2477026d8d4fac945714c894cca485 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/3d2477026d8d4fac945714c894cca485
I20250112 20:57:50.028616 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 3d2477026d8d4fac945714c894cca485 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:50.029486 17427 tablet_replica.cc:331] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:57:50.030591 17427 raft_consensus.cc:2238] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:50.031227 17427 raft_consensus.cc:2267] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:50.032846 16532 catalog_manager.cc:5526] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:50.043332 17427 ts_tablet_manager.cc:1905] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:50.060972 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ffbc5c73802406893d78f89cf5c145c with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:57:50.069890 17426 tablet_replica.cc:331] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:57:50.070895 17426 raft_consensus.cc:2238] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:50.071514 17426 raft_consensus.cc:2267] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:50.074545 17427 ts_tablet_manager.cc:1918] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:50.074661 17426 ts_tablet_manager.cc:1905] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:50.075084 17427 log.cc:1198] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/3e3b5e20d22c4251b1e2b91ca0479e2f
I20250112 20:57:50.080530 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 3e3b5e20d22c4251b1e2b91ca0479e2f (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:50.091553 17426 ts_tablet_manager.cc:1918] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:50.092033 17426 log.cc:1198] T 5ffbc5c73802406893d78f89cf5c145c P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/5ffbc5c73802406893d78f89cf5c145c
I20250112 20:57:50.094393 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 5ffbc5c73802406893d78f89cf5c145c (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:51.251027 16709 consensus_queue.cc:237] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:51.301234 16782 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:51.302414 17098 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:51.308849 16638 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:51.330170 17257 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.337769 16782 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:57:51.343644 16678 consensus_peers.cc:487] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd7c3a6eea9043f4b438c4e28aca966e. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:51.346812 16634 consensus_queue.cc:237] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:51.348001 16537 catalog_manager.cc:5526] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:51.353139 17257 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:51.368767 16637 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:57:51.385762 16603 consensus_peers.cc:487] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 132239c0c07a4c2f9334e84188f329b3. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:51.389210 16782 raft_consensus.cc:1270] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:51.393244 17430 consensus_queue.cc:1035] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:51.397756 16709 raft_consensus.cc:1270] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:51.400722 17431 consensus_queue.cc:1035] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:51.413079 17431 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.414403 16783 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.429153 16785 consensus_queue.cc:237] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:51.439072 16532 catalog_manager.cc:5526] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.441273 16709 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:57:51.453709 16752 consensus_peers.cc:487] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e510cf639924856a759b20574cb60be. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:51.454164 16709 raft_consensus.cc:1270] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:51.455500 17437 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:51.459728 16634 raft_consensus.cc:1270] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:51.461228 17437 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:51.464720 17261 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.469442 16714 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.498186 16537 catalog_manager.cc:5526] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.501718 16708 consensus_queue.cc:237] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:51.513859 16637 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:57:51.519819 16678 consensus_peers.cc:487] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f63edc7cd344fc88b39d2a6b2fbceb. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:51.522382 16785 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:51.524353 17428 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:51.529644 16634 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:51.531033 17428 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:51.538779 17428 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.539026 16782 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.558037 16537 catalog_manager.cc:5526] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.563361 16638 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:51.747581 17458 ts_tablet_manager.cc:927] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:51.753752 17458 tablet_copy_client.cc:323] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:57:51.795245 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet dd7c3a6eea9043f4b438c4e28aca966e from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:57:51.795832 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet dd7c3a6eea9043f4b438c4e28aca966e from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-dd7c3a6eea9043f4b438c4e28aca966e
I20250112 20:57:51.806150 16724 tablet_copy_source_session.cc:215] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:51.808971 17458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd7c3a6eea9043f4b438c4e28aca966e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:51.827730 17458 tablet_copy_client.cc:806] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:51.828400 17458 tablet_copy_client.cc:670] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:51.832487 17458 tablet_copy_client.cc:538] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:51.840435 17458 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:51.864965 17458 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:51.865934 17458 tablet_bootstrap.cc:492] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:51.866607 17458 ts_tablet_manager.cc:1397] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.026s user 0.013s sys 0.007s
I20250112 20:57:51.869171 17458 raft_consensus.cc:357] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:51.870163 17458 raft_consensus.cc:738] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:51.870733 17458 consensus_queue.cc:260] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:51.873817 17458 ts_tablet_manager.cc:1428] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:57:51.875190 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-dd7c3a6eea9043f4b438c4e28aca966e received from {username='slave'} at 127.0.0.1:48112
I20250112 20:57:51.875655 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-dd7c3a6eea9043f4b438c4e28aca966e on tablet dd7c3a6eea9043f4b438c4e28aca966e with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:51.881626 17458 ts_tablet_manager.cc:927] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:51.885411 17458 tablet_copy_client.cc:323] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:57:51.886672 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 132239c0c07a4c2f9334e84188f329b3 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:57:51.887159 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 132239c0c07a4c2f9334e84188f329b3 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-132239c0c07a4c2f9334e84188f329b3
I20250112 20:57:51.906716 16649 tablet_copy_source_session.cc:215] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:51.909231 17458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 132239c0c07a4c2f9334e84188f329b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:51.921945 17458 tablet_copy_client.cc:806] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:51.922626 17458 tablet_copy_client.cc:670] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:51.925478 17458 tablet_copy_client.cc:538] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:51.934298 17458 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:51.952945 17458 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:51.955698 17458 tablet_bootstrap.cc:492] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:51.956339 17458 ts_tablet_manager.cc:1397] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.010s
I20250112 20:57:51.958575 17458 raft_consensus.cc:357] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:51.959429 17458 raft_consensus.cc:738] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:51.960047 17458 consensus_queue.cc:260] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:51.962284 17458 ts_tablet_manager.cc:1428] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:57:51.964044 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-132239c0c07a4c2f9334e84188f329b3 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:57:51.964430 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-132239c0c07a4c2f9334e84188f329b3 on tablet 132239c0c07a4c2f9334e84188f329b3 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:51.973968 17458 ts_tablet_manager.cc:927] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:51.975857 17458 tablet_copy_client.cc:323] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:57:51.977139 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet 0e510cf639924856a759b20574cb60be from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:57:51.977669 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet 0e510cf639924856a759b20574cb60be from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-0e510cf639924856a759b20574cb60be
I20250112 20:57:51.982759 16798 tablet_copy_source_session.cc:215] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:51.985188 17458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e510cf639924856a759b20574cb60be. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:51.997218 17458 tablet_copy_client.cc:806] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:51.997813 17458 tablet_copy_client.cc:670] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:52.000896 17458 tablet_copy_client.cc:538] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:52.008813 17458 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:52.025301 17458 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:52.025883 17458 tablet_bootstrap.cc:492] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:52.026343 17458 ts_tablet_manager.cc:1397] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.008s
I20250112 20:57:52.028170 17458 raft_consensus.cc:357] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:52.028822 17458 raft_consensus.cc:738] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:52.029349 17458 consensus_queue.cc:260] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:52.033051 17458 ts_tablet_manager.cc:1428] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20250112 20:57:52.034328 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-0e510cf639924856a759b20574cb60be received from {username='slave'} at 127.0.0.1:51714
I20250112 20:57:52.034629 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-0e510cf639924856a759b20574cb60be on tablet 0e510cf639924856a759b20574cb60be with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:52.037290 17468 ts_tablet_manager.cc:927] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:52.053303 17468 tablet_copy_client.cc:323] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:57:52.054703 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 43f63edc7cd344fc88b39d2a6b2fbceb from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:57:52.055107 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 43f63edc7cd344fc88b39d2a6b2fbceb from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-43f63edc7cd344fc88b39d2a6b2fbceb
I20250112 20:57:52.061223 16724 tablet_copy_source_session.cc:215] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:52.063959 17468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43f63edc7cd344fc88b39d2a6b2fbceb. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:57:52.073438 16568 debug-util.cc:398] Leaking SignalData structure 0x7b0800161320 after lost signal to thread 15457
W20250112 20:57:52.074383 16568 debug-util.cc:398] Leaking SignalData structure 0x7b08001f7600 after lost signal to thread 16571
W20250112 20:57:52.075481 16568 debug-util.cc:398] Leaking SignalData structure 0x7b08001de3c0 after lost signal to thread 16663
W20250112 20:57:52.076267 16568 debug-util.cc:398] Leaking SignalData structure 0x7b0800161540 after lost signal to thread 16738
W20250112 20:57:52.077755 16568 debug-util.cc:398] Leaking SignalData structure 0x7b080005aa40 after lost signal to thread 16812
I20250112 20:57:52.090790 17468 tablet_copy_client.cc:806] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:52.091343 17468 tablet_copy_client.cc:670] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:52.100694 17468 tablet_copy_client.cc:538] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:52.110016 17468 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:52.152770 17468 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:52.153684 17468 tablet_bootstrap.cc:492] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:52.154343 17468 ts_tablet_manager.cc:1397] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.045s user 0.018s sys 0.005s
I20250112 20:57:52.157100 17468 raft_consensus.cc:357] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:52.158118 17468 raft_consensus.cc:738] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:52.158807 17468 consensus_queue.cc:260] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:52.161906 17468 ts_tablet_manager.cc:1428] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20250112 20:57:52.163347 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-43f63edc7cd344fc88b39d2a6b2fbceb received from {username='slave'} at 127.0.0.1:48112
I20250112 20:57:52.163763 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-43f63edc7cd344fc88b39d2a6b2fbceb on tablet 43f63edc7cd344fc88b39d2a6b2fbceb with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:52.282516 17324 raft_consensus.cc:1212] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:52.385630 17324 raft_consensus.cc:1212] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:52.464506 17324 raft_consensus.cc:1212] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:52.602018 17324 raft_consensus.cc:1212] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:52.833977 17098 raft_consensus.cc:1059] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:52.835706 17098 consensus_queue.cc:237] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:52.844086 17324 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:52.845640 16634 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:52.847265 17257 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:52.848789 17257 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:52.867126 16782 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:52.869025 17098 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:52.880683 17098 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:52.881788 16782 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:52.884500 16637 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:52.899274 17324 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:52.903904 16534 catalog_manager.cc:5526] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:57:52.948905 16709 consensus_queue.cc:237] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:52.957731 16785 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:52.960083 17257 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:52.964476 17439 raft_consensus.cc:1059] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:52.987371 17439 consensus_queue.cc:237] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:53.010788 17324 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:53.102334 17324 raft_consensus.cc:1270] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.103626 17471 consensus_queue.cc:1035] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:53.108517 16714 raft_consensus.cc:1270] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.112876 17439 consensus_queue.cc:1035] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:53.118716 17098 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:53.131181 17324 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.129662 17098 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.147603 17430 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.166636 16709 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.176523 17483 raft_consensus.cc:1059] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:53.181448 17483 consensus_queue.cc:237] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:53.191417 16782 raft_consensus.cc:2949] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
W20250112 20:57:53.210544 16751 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:38087, user_credentials={real_user=slave}} blocked reactor thread for 50090.1us
I20250112 20:57:53.215579 16532 catalog_manager.cc:5526] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.219260 16709 raft_consensus.cc:1270] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.221949 17323 raft_consensus.cc:1270] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.223033 17493 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:53.241418 17261 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:53.250288 17432 raft_consensus.cc:1059] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:53.253558 17432 consensus_queue.cc:237] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:53.269301 17486 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
W20250112 20:57:53.307128 16602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:38087, user_credentials={real_user=slave}} blocked reactor thread for 130121us
I20250112 20:57:53.335470 16638 raft_consensus.cc:1270] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.359120 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet dd7c3a6eea9043f4b438c4e28aca966e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:53.362840 17324 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.367645 17322 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.367650 16532 catalog_manager.cc:5526] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.383291 17321 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.387303 16637 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.396687 16782 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.411012 16785 raft_consensus.cc:1270] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:53.426538 17504 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:53.428985 17432 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:53.442822 17483 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:53.510299 17432 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:53.505025 16707 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.527812 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet dd7c3a6eea9043f4b438c4e28aca966e with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:57:53.552148 17439 consensus_queue.cc:1035] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:53.580056 16635 consensus_queue.cc:237] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:53.605406 16714 raft_consensus.cc:1270] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:53.607892 17487 consensus_queue.cc:1035] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:57:53.627574 17321 raft_consensus.cc:1270] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:53.637693 17487 consensus_queue.cc:1035] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:53.638779 16632 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.631135 16532 catalog_manager.cc:5526] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:57:53.655306 17471 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.662680 17507 tablet_replica.cc:331] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:57:53.665619 17321 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.668097 16707 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.681386 16534 catalog_manager.cc:5526] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.684681 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 132239c0c07a4c2f9334e84188f329b3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20250112 20:57:53.698223 16678 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:41463, user_credentials={real_user=slave}} blocked reactor thread for 45308.6us
I20250112 20:57:53.705443 17496 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.719792 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 132239c0c07a4c2f9334e84188f329b3 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:57:53.726421 16782 raft_consensus.cc:2949] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.732620 16637 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.745155 16785 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.752113 16532 catalog_manager.cc:5526] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:57:53.756992 17507 raft_consensus.cc:2238] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:53.757817 17507 raft_consensus.cc:2267] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:53.761803 17321 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.771224 17511 tablet_replica.cc:331] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:57:53.772521 17511 raft_consensus.cc:2238] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:53.773144 17511 raft_consensus.cc:2267] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:53.778779 17507 ts_tablet_manager.cc:1905] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:53.783644 17511 ts_tablet_manager.cc:1905] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:53.796679 16707 consensus_queue.cc:237] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:53.807245 17507 ts_tablet_manager.cc:1918] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:53.808064 17507 log.cc:1198] T dd7c3a6eea9043f4b438c4e28aca966e P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/dd7c3a6eea9043f4b438c4e28aca966e
I20250112 20:57:53.809978 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet dd7c3a6eea9043f4b438c4e28aca966e (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:53.813937 17321 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:53.814841 16782 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:53.814379 17511 ts_tablet_manager.cc:1918] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:53.815707 17511 log.cc:1198] T 132239c0c07a4c2f9334e84188f329b3 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/132239c0c07a4c2f9334e84188f329b3
I20250112 20:57:53.816115 17496 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:53.818559 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet 132239c0c07a4c2f9334e84188f329b3 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:53.818872 17482 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:57:53.828842 17496 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.830330 17322 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.832805 16785 raft_consensus.cc:2949] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.840425 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 43f63edc7cd344fc88b39d2a6b2fbceb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:53.843874 16534 catalog_manager.cc:5526] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.859434 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 43f63edc7cd344fc88b39d2a6b2fbceb with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:57:53.860191 17507 tablet_replica.cc:331] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:57:53.861248 17507 raft_consensus.cc:2238] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:53.861905 17507 raft_consensus.cc:2267] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:53.867048 17507 ts_tablet_manager.cc:1905] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:53.880187 17507 ts_tablet_manager.cc:1918] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:53.880661 17507 log.cc:1198] T 43f63edc7cd344fc88b39d2a6b2fbceb P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/43f63edc7cd344fc88b39d2a6b2fbceb
I20250112 20:57:53.882025 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 43f63edc7cd344fc88b39d2a6b2fbceb (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:53.964217 16785 consensus_queue.cc:237] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:53.968106 17322 raft_consensus.cc:1270] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:53.968720 16637 raft_consensus.cc:1270] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:53.969870 17437 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:53.970742 17490 consensus_queue.cc:1035] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:53.978410 17490 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.979913 16637 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.988775 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 0e510cf639924856a759b20574cb60be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:53.997067 17322 raft_consensus.cc:2949] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:53.995265 16534 catalog_manager.cc:5526] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:54.007510 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e510cf639924856a759b20574cb60be with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:57:54.009393 17512 tablet_replica.cc:331] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:57:54.022432 17512 raft_consensus.cc:2238] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:54.023031 17512 raft_consensus.cc:2267] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:54.025717 17512 ts_tablet_manager.cc:1905] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:54.035686 17512 ts_tablet_manager.cc:1918] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:54.036062 17512 log.cc:1198] T 0e510cf639924856a759b20574cb60be P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/0e510cf639924856a759b20574cb60be
I20250112 20:57:54.037441 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 0e510cf639924856a759b20574cb60be (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
W20250112 20:57:54.156059 16809 debug-util.cc:398] Leaking SignalData structure 0x7b0800198f40 after lost signal to thread 15457
W20250112 20:57:54.157776 16809 debug-util.cc:398] Leaking SignalData structure 0x7b080013fbe0 after lost signal to thread 16571
W20250112 20:57:54.159080 16809 debug-util.cc:398] Leaking SignalData structure 0x7b08001b4220 after lost signal to thread 16663
W20250112 20:57:54.160315 16809 debug-util.cc:398] Leaking SignalData structure 0x7b080016c700 after lost signal to thread 16738
W20250112 20:57:54.161396 16809 debug-util.cc:398] Leaking SignalData structure 0x7b08002d0c80 after lost signal to thread 16812
W20250112 20:57:54.162168 16809 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd420 after lost signal to thread 17349
I20250112 20:57:55.091729 16714 consensus_queue.cc:237] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.099104 16637 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250112 20:57:55.099941 16678 consensus_peers.cc:487] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 142fbcc35085432abdb80a4fad031c67. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:55.099998 16782 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:55.100904 17496 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:55.101800 17257 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:55.110319 17496 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.117180 16637 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.123396 16782 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.135743 16785 consensus_queue.cc:237] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.142735 16534 catalog_manager.cc:5526] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250112 20:57:55.150955 16752 consensus_peers.cc:487] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0f35368d307429d8a87ba0c17bb4caf. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:55.151289 16714 raft_consensus.cc:1270] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:55.152760 17489 consensus_queue.cc:1035] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:55.155570 16634 raft_consensus.cc:1270] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:55.156908 17490 consensus_queue.cc:1035] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:55.206508 17490 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.222967 16714 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.250183 16634 consensus_queue.cc:237] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.254951 16532 catalog_manager.cc:5526] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.274578 16708 raft_consensus.cc:1270] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250112 20:57:55.275635 16603 consensus_peers.cc:487] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e19f4890a44f4f088549ad693bafbf8d. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:55.276757 17471 consensus_queue.cc:1035] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:57:55.279836 16752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:41463, user_credentials={real_user=slave}} blocked reactor thread for 71272.1us
I20250112 20:57:55.280148 16785 raft_consensus.cc:1270] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:55.287225 17522 consensus_queue.cc:1035] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:55.289758 16634 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.316002 17513 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.333631 16714 consensus_queue.cc:237] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.334385 16708 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.342649 16785 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:57:55.362886 16678 consensus_peers.cc:487] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc2820c3d5974dfcb2230e18411e6493. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:55.371405 16634 raft_consensus.cc:1270] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:55.374164 17496 consensus_queue.cc:1035] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:55.380805 16537 catalog_manager.cc:5526] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:55.387537 16782 raft_consensus.cc:1270] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:55.391189 17257 consensus_queue.cc:1035] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:55.394759 17257 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.400621 16637 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.428047 16785 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.423969 16532 catalog_manager.cc:5526] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:55.649580 17539 ts_tablet_manager.cc:927] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:55.659178 17539 tablet_copy_client.cc:323] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:57:55.661775 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet e19f4890a44f4f088549ad693bafbf8d from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:57:55.662351 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet e19f4890a44f4f088549ad693bafbf8d from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-e19f4890a44f4f088549ad693bafbf8d
I20250112 20:57:55.666546 17541 ts_tablet_manager.cc:927] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:55.670540 16649 tablet_copy_source_session.cc:215] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:55.674108 17539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e19f4890a44f4f088549ad693bafbf8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:55.697973 17539 tablet_copy_client.cc:806] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:55.698748 17539 tablet_copy_client.cc:670] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:55.702764 17539 tablet_copy_client.cc:538] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:55.712131 17541 tablet_copy_client.cc:323] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:57:55.713830 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 142fbcc35085432abdb80a4fad031c67 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:57:55.714239 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 142fbcc35085432abdb80a4fad031c67 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-142fbcc35085432abdb80a4fad031c67
I20250112 20:57:55.723268 16724 tablet_copy_source_session.cc:215] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:55.725978 17541 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 142fbcc35085432abdb80a4fad031c67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:55.728575 17539 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:55.758826 17539 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:55.759733 17539 tablet_bootstrap.cc:492] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:55.760408 17539 ts_tablet_manager.cc:1397] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.016s
I20250112 20:57:55.763096 17539 raft_consensus.cc:357] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.763968 17539 raft_consensus.cc:738] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:55.764549 17539 consensus_queue.cc:260] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.765386 17544 ts_tablet_manager.cc:927] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:55.775466 17544 tablet_copy_client.cc:323] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:57:55.777436 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet b0f35368d307429d8a87ba0c17bb4caf from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:57:55.777971 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet b0f35368d307429d8a87ba0c17bb4caf from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-b0f35368d307429d8a87ba0c17bb4caf
I20250112 20:57:55.786821 16798 tablet_copy_source_session.cc:215] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:55.792335 17544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0f35368d307429d8a87ba0c17bb4caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:55.799381 17541 tablet_copy_client.cc:806] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:55.802177 17539 ts_tablet_manager.cc:1428] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.041s user 0.005s sys 0.003s
I20250112 20:57:55.803328 17541 tablet_copy_client.cc:670] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:55.809988 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-e19f4890a44f4f088549ad693bafbf8d received from {username='slave'} at 127.0.0.1:36254
I20250112 20:57:55.810428 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-e19f4890a44f4f088549ad693bafbf8d on tablet e19f4890a44f4f088549ad693bafbf8d with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:55.833042 17541 tablet_copy_client.cc:538] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:55.835914 17544 tablet_copy_client.cc:806] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:55.836511 17544 tablet_copy_client.cc:670] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:55.840530 17544 tablet_copy_client.cc:538] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:55.847993 17541 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:55.853117 17539 ts_tablet_manager.cc:927] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:55.860113 17544 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:55.868681 17539 tablet_copy_client.cc:323] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:57:55.871759 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet cc2820c3d5974dfcb2230e18411e6493 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:57:55.872227 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet cc2820c3d5974dfcb2230e18411e6493 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-cc2820c3d5974dfcb2230e18411e6493
I20250112 20:57:55.875173 17541 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:55.875981 17541 tablet_bootstrap.cc:492] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:55.879572 16724 tablet_copy_source_session.cc:215] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:55.879858 17541 ts_tablet_manager.cc:1397] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.005s
I20250112 20:57:55.882725 17541 raft_consensus.cc:357] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.883697 17541 raft_consensus.cc:738] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:55.884371 17541 consensus_queue.cc:260] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.887177 17541 ts_tablet_manager.cc:1428] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:57:55.889132 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-142fbcc35085432abdb80a4fad031c67 received from {username='slave'} at 127.0.0.1:48112
I20250112 20:57:55.889513 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-142fbcc35085432abdb80a4fad031c67 on tablet 142fbcc35085432abdb80a4fad031c67 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:55.944613 17539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc2820c3d5974dfcb2230e18411e6493. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:55.980857 17544 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:55.981724 17544 tablet_bootstrap.cc:492] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:55.982313 17544 ts_tablet_manager.cc:1397] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.124s user 0.044s sys 0.032s
I20250112 20:57:55.984194 17539 tablet_copy_client.cc:806] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:55.986709 17539 tablet_copy_client.cc:670] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:55.985105 17544 raft_consensus.cc:357] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.987241 17544 raft_consensus.cc:738] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:55.988009 17544 consensus_queue.cc:260] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:55.991652 17544 ts_tablet_manager.cc:1428] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20250112 20:57:55.993445 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-b0f35368d307429d8a87ba0c17bb4caf received from {username='slave'} at 127.0.0.1:51714
I20250112 20:57:55.993836 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-b0f35368d307429d8a87ba0c17bb4caf on tablet b0f35368d307429d8a87ba0c17bb4caf with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:55.995874 17539 tablet_copy_client.cc:538] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:56.004961 17539 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:56.020625 17539 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:56.021328 17539 tablet_bootstrap.cc:492] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:56.021867 17539 ts_tablet_manager.cc:1397] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20250112 20:57:56.024135 17539 raft_consensus.cc:357] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:56.025015 17539 raft_consensus.cc:738] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:56.025640 17539 consensus_queue.cc:260] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:56.028316 17539 ts_tablet_manager.cc:1428] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:57:56.029693 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-cc2820c3d5974dfcb2230e18411e6493 received from {username='slave'} at 127.0.0.1:48112
I20250112 20:57:56.030055 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-cc2820c3d5974dfcb2230e18411e6493 on tablet cc2820c3d5974dfcb2230e18411e6493 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:56.141280 17322 raft_consensus.cc:1212] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:56.197337 17322 raft_consensus.cc:1212] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:56.384990 17322 raft_consensus.cc:1212] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:56.385713 17321 raft_consensus.cc:1212] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:56.618160 17257 raft_consensus.cc:1059] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:56.620234 17257 consensus_queue.cc:237] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:56.626184 16632 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.628886 17321 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.628908 17257 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:56.632701 17257 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:57:56.635704 16783 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.637967 17515 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:56.652254 17534 raft_consensus.cc:1059] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:56.654148 17496 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.654789 17534 consensus_queue.cc:237] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:56.657001 16785 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.656652 16632 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.665192 17321 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.678153 16537 catalog_manager.cc:5526] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:57:56.683645 17324 raft_consensus.cc:1270] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.686530 17522 consensus_queue.cc:1035] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:56.693099 16714 raft_consensus.cc:1270] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.693753 16785 raft_consensus.cc:1270] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.695025 17534 consensus_queue.cc:1035] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:56.695983 17513 consensus_queue.cc:1035] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:56.708213 17513 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.710954 16783 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.721483 17324 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.740504 16714 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.766657 16534 catalog_manager.cc:5526] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.763300 16714 consensus_queue.cc:237] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:56.781515 16637 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:56.783160 17515 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:56.789906 17257 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.791275 16634 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.798761 17324 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:56.800499 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 142fbcc35085432abdb80a4fad031c67 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:56.801044 17515 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:56.815429 16637 consensus_queue.cc:237] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:56.821892 16783 raft_consensus.cc:1270] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:56.823983 17321 raft_consensus.cc:1270] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:56.825210 17522 consensus_queue.cc:1035] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:56.826746 17534 consensus_queue.cc:1035] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:56.818555 16533 catalog_manager.cc:5526] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:57:56.828292 17324 raft_consensus.cc:2949] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.841739 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 142fbcc35085432abdb80a4fad031c67 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:57:56.838340 17534 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.839682 17530 raft_consensus.cc:1059] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:56.846855 17530 consensus_queue.cc:237] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:56.843492 17322 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.853535 17566 tablet_replica.cc:331] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:57:56.854542 17566 raft_consensus.cc:2238] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:56.855240 17566 raft_consensus.cc:2267] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:56.858839 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet e19f4890a44f4f088549ad693bafbf8d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:56.858405 17566 ts_tablet_manager.cc:1905] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:56.862454 16537 catalog_manager.cc:5526] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.863926 16783 raft_consensus.cc:2949] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.873765 17322 raft_consensus.cc:1270] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.874559 16709 raft_consensus.cc:1270] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.875982 17553 consensus_queue.cc:1035] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:56.878748 17566 ts_tablet_manager.cc:1918] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:56.879202 17566 log.cc:1198] T 142fbcc35085432abdb80a4fad031c67 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/142fbcc35085432abdb80a4fad031c67
I20250112 20:57:56.880360 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet e19f4890a44f4f088549ad693bafbf8d with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:57:56.881723 17529 consensus_queue.cc:1035] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:56.895972 17570 tablet_replica.cc:331] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:57:56.898058 17570 raft_consensus.cc:2238] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:56.899838 17570 raft_consensus.cc:2267] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:56.905472 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet 142fbcc35085432abdb80a4fad031c67 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:56.908059 16637 raft_consensus.cc:1270] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:56.916435 17570 ts_tablet_manager.cc:1905] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:56.926625 17553 consensus_queue.cc:1035] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:56.929155 17553 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.931995 17322 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.936261 17570 ts_tablet_manager.cc:1918] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:56.936795 17570 log.cc:1198] T e19f4890a44f4f088549ad693bafbf8d P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/e19f4890a44f4f088549ad693bafbf8d
I20250112 20:57:56.938395 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet e19f4890a44f4f088549ad693bafbf8d (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:56.945919 16709 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.948354 16537 catalog_manager.cc:5526] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.962397 16634 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.976899 17496 raft_consensus.cc:1059] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:56.977643 16785 consensus_queue.cc:237] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:56.978924 17496 consensus_queue.cc:237] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:56.981585 16714 raft_consensus.cc:1270] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:56.982710 17568 consensus_queue.cc:1035] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:56.985073 17322 raft_consensus.cc:1270] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:56.992889 17568 consensus_queue.cc:1035] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:56.992318 17530 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.997597 16709 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:56.999249 17322 raft_consensus.cc:2949] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.003556 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet b0f35368d307429d8a87ba0c17bb4caf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:57.014974 16537 catalog_manager.cc:5526] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.019747 16783 raft_consensus.cc:1270] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:57.021036 16634 raft_consensus.cc:1270] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:57.020321 17322 raft_consensus.cc:1270] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:57.028609 17515 consensus_queue.cc:1035] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:57.030416 17496 consensus_queue.cc:1035] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:57.046500 17257 consensus_queue.cc:1035] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:57.049872 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet b0f35368d307429d8a87ba0c17bb4caf with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:57:57.059607 17515 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.060788 17578 tablet_replica.cc:331] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:57:57.061856 16785 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.062913 17578 raft_consensus.cc:2238] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:57.063720 17578 raft_consensus.cc:2267] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:57.072117 16634 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.074146 17578 ts_tablet_manager.cc:1905] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:57.080053 17322 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.081830 16537 catalog_manager.cc:5526] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.098148 17578 ts_tablet_manager.cc:1918] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:57.098713 17578 log.cc:1198] T b0f35368d307429d8a87ba0c17bb4caf P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/b0f35368d307429d8a87ba0c17bb4caf
I20250112 20:57:57.101306 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet b0f35368d307429d8a87ba0c17bb4caf (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:57.110368 16707 consensus_queue.cc:237] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:57.115036 17322 raft_consensus.cc:1270] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:57.115128 16637 raft_consensus.cc:1270] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:57.116380 17257 consensus_queue.cc:1035] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:57.117101 17574 consensus_queue.cc:1035] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:57.122509 17257 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.126255 16637 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.130136 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet cc2820c3d5974dfcb2230e18411e6493 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:57.132023 17322 raft_consensus.cc:2949] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:57.140899 16532 catalog_manager.cc:5526] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:57:57.152526 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet cc2820c3d5974dfcb2230e18411e6493 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:57:57.153318 17566 tablet_replica.cc:331] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:57:57.154130 17566 raft_consensus.cc:2238] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:57:57.154574 17566 raft_consensus.cc:2267] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:57:57.157029 17566 ts_tablet_manager.cc:1905] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:57:57.167503 17566 ts_tablet_manager.cc:1918] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:57:57.167888 17566 log.cc:1198] T cc2820c3d5974dfcb2230e18411e6493 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/cc2820c3d5974dfcb2230e18411e6493
I20250112 20:57:57.169173 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet cc2820c3d5974dfcb2230e18411e6493 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:57:58.205677 16707 consensus_queue.cc:237] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
W20250112 20:57:58.212038 16678 consensus_peers.cc:487] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4799def22f754c9881fd8245eb8f75b3. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:58.212618 16783 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:58.214052 16634 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:58.214085 17575 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:58.215991 17575 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:58.235476 16634 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.233309 17575 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.243254 16785 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.252686 16534 catalog_manager.cc:5526] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.263234 16783 consensus_queue.cc:237] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.275458 16707 raft_consensus.cc:1270] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:58.279495 17589 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:57:58.280534 16752 consensus_peers.cc:487] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f61fd1d8bf44f64b73ade11c0224df0. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:58.292276 16637 raft_consensus.cc:1270] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:58.296392 17530 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:58.300282 17553 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.303392 16707 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.316819 16634 consensus_queue.cc:237] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.316571 16637 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.319093 16532 catalog_manager.cc:5526] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.333640 16714 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:58.334378 16783 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:58.335080 17534 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:58.336700 17534 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:57:58.338852 16603 consensus_peers.cc:487] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c10f3f7e0f74c9f988fa01e7843e961. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:58.345041 17594 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.352447 16714 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.358073 16783 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.371165 16532 catalog_manager.cc:5526] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.374078 16783 consensus_queue.cc:237] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.387562 16637 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250112 20:57:58.388473 16752 consensus_peers.cc:487] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 44e603d3a09141d296df5d386a3b4d11. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:57:58.389206 16714 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:57:58.389232 17589 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:57:58.390244 17530 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:57:58.398962 17553 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.400558 16637 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.401043 16714 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.416170 16532 catalog_manager.cc:5526] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:57:58.722034 17600 ts_tablet_manager.cc:927] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:58.724654 17600 tablet_copy_client.cc:323] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:57:58.726054 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet 1f61fd1d8bf44f64b73ade11c0224df0 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:57:58.726490 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet 1f61fd1d8bf44f64b73ade11c0224df0 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-1f61fd1d8bf44f64b73ade11c0224df0
I20250112 20:57:58.732990 16798 tablet_copy_source_session.cc:215] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:58.735605 17600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f61fd1d8bf44f64b73ade11c0224df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:58.750404 17600 tablet_copy_client.cc:806] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:58.750897 17600 tablet_copy_client.cc:670] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:58.756009 17600 tablet_copy_client.cc:538] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:58.764134 17600 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:58.786063 17603 ts_tablet_manager.cc:927] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:57:58.788039 17603 tablet_copy_client.cc:323] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:57:58.789404 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 8c10f3f7e0f74c9f988fa01e7843e961 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:57:58.789896 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 8c10f3f7e0f74c9f988fa01e7843e961 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-8c10f3f7e0f74c9f988fa01e7843e961
I20250112 20:57:58.794219 17600 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:58.796615 17600 tablet_bootstrap.cc:492] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:58.797297 17600 ts_tablet_manager.cc:1397] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.024s
I20250112 20:57:58.798424 16649 tablet_copy_source_session.cc:215] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:58.800563 17600 raft_consensus.cc:357] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.801453 17600 raft_consensus.cc:738] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:58.803339 17600 consensus_queue.cc:260] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.806301 17603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c10f3f7e0f74c9f988fa01e7843e961. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:58.826136 17600 ts_tablet_manager.cc:1428] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.029s user 0.016s sys 0.002s
I20250112 20:57:58.828014 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-1f61fd1d8bf44f64b73ade11c0224df0 received from {username='slave'} at 127.0.0.1:51714
I20250112 20:57:58.828449 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-1f61fd1d8bf44f64b73ade11c0224df0 on tablet 1f61fd1d8bf44f64b73ade11c0224df0 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:58.831949 17603 tablet_copy_client.cc:806] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:58.832587 17603 tablet_copy_client.cc:670] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:58.832103 17600 ts_tablet_manager.cc:927] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:57:58.841768 17603 tablet_copy_client.cc:538] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:58.842072 17600 tablet_copy_client.cc:323] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:57:58.844700 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 4799def22f754c9881fd8245eb8f75b3 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:57:58.845227 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 4799def22f754c9881fd8245eb8f75b3 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-4799def22f754c9881fd8245eb8f75b3
I20250112 20:57:58.848664 17603 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:58.852517 16724 tablet_copy_source_session.cc:215] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:58.855595 17600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4799def22f754c9881fd8245eb8f75b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:58.874404 17600 tablet_copy_client.cc:806] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:58.875118 17600 tablet_copy_client.cc:670] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:58.874544 17606 ts_tablet_manager.cc:927] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:57:58.878372 17603 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:58.879297 17603 tablet_bootstrap.cc:492] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:58.879925 17603 ts_tablet_manager.cc:1397] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.011s
I20250112 20:57:58.880914 17600 tablet_copy_client.cc:538] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:58.882833 17603 raft_consensus.cc:357] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.884863 17603 raft_consensus.cc:738] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:58.885494 17603 consensus_queue.cc:260] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.889041 17603 ts_tablet_manager.cc:1428] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20250112 20:57:58.890878 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-8c10f3f7e0f74c9f988fa01e7843e961 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:57:58.891343 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-8c10f3f7e0f74c9f988fa01e7843e961 on tablet 8c10f3f7e0f74c9f988fa01e7843e961 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:58.891767 17600 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:58.917405 17606 tablet_copy_client.cc:323] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:57:58.919486 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet 44e603d3a09141d296df5d386a3b4d11 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:57:58.919931 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet 44e603d3a09141d296df5d386a3b4d11 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-44e603d3a09141d296df5d386a3b4d11
I20250112 20:57:58.927088 17600 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:58.927803 17600 tablet_bootstrap.cc:492] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:58.928464 17600 ts_tablet_manager.cc:1397] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.037s user 0.006s sys 0.013s
I20250112 20:57:58.931932 16798 tablet_copy_source_session.cc:215] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:57:58.931463 17600 raft_consensus.cc:357] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.932627 17600 raft_consensus.cc:738] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:58.934782 17600 consensus_queue.cc:260] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.937515 17600 ts_tablet_manager.cc:1428] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250112 20:57:58.938064 17606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44e603d3a09141d296df5d386a3b4d11. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:57:58.939035 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-4799def22f754c9881fd8245eb8f75b3 received from {username='slave'} at 127.0.0.1:48112
I20250112 20:57:58.939451 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-4799def22f754c9881fd8245eb8f75b3 on tablet 4799def22f754c9881fd8245eb8f75b3 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:58.957461 17606 tablet_copy_client.cc:806] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:57:58.957999 17606 tablet_copy_client.cc:670] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:57:58.962036 17606 tablet_copy_client.cc:538] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:57:58.971129 17606 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:57:58.995361 17606 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:57:58.996038 17606 tablet_bootstrap.cc:492] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:57:58.996510 17606 ts_tablet_manager.cc:1397] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.007s
I20250112 20:57:58.998726 17606 raft_consensus.cc:357] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:58.999333 17606 raft_consensus.cc:738] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:57:58.999850 17606 consensus_queue.cc:260] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:57:59.003000 17606 ts_tablet_manager.cc:1428] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:57:59.004583 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-44e603d3a09141d296df5d386a3b4d11 received from {username='slave'} at 127.0.0.1:51714
I20250112 20:57:59.005024 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-44e603d3a09141d296df5d386a3b4d11 on tablet 44e603d3a09141d296df5d386a3b4d11 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:57:59.235076 17322 raft_consensus.cc:1212] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:59.260259 17322 raft_consensus.cc:1212] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:59.269238 17324 raft_consensus.cc:1212] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:59.421785 17324 raft_consensus.cc:1212] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:57:59.719280 17534 raft_consensus.cc:1059] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:59.721300 17534 consensus_queue.cc:237] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:59.726384 16783 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.726467 17324 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.727186 16707 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.728061 17580 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.728937 17594 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.730367 17617 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.740587 17614 raft_consensus.cc:1059] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:59.750417 17580 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.751562 16783 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.743502 17614 consensus_queue.cc:237] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:59.771975 17324 raft_consensus.cc:1270] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.772066 16707 raft_consensus.cc:1270] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.773638 17614 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.776249 16714 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.776635 16534 catalog_manager.cc:5526] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.780368 17614 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.790055 17322 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.794087 17612 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.795428 16634 raft_consensus.cc:1270] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.803815 17613 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.818049 16714 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.825428 17322 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.834345 16634 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.840330 17575 raft_consensus.cc:1059] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:59.842758 17575 consensus_queue.cc:237] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:59.845012 17614 raft_consensus.cc:1059] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:57:59.848126 17614 consensus_queue.cc:237] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:59.857420 16637 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.858314 17324 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.859001 17584 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.861083 16782 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:57:59.874725 17584 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.877610 17584 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:57:59.894876 16632 consensus_queue.cc:237] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:59.902287 16537 catalog_manager.cc:5526] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.921420 17324 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:59.927467 16785 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:59.927803 17594 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:57:59.931537 16785 consensus_queue.cc:237] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:57:59.933087 17617 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:59.937898 17534 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.942309 17321 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.960137 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 8c10f3f7e0f74c9f988fa01e7843e961 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:57:59.963126 17575 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.965385 17322 raft_consensus.cc:1270] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:59.964808 16783 raft_consensus.cc:2949] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.970383 16708 raft_consensus.cc:1270] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:57:59.970629 17625 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:57:59.978160 17613 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.968879 16537 catalog_manager.cc:5526] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:57:59.983407 17324 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.983153 17322 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.993427 17625 consensus_queue.cc:1035] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:57:59.995539 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c10f3f7e0f74c9f988fa01e7843e961 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:57:59.998059 16637 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:57:59.998430 16707 raft_consensus.cc:2949] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.008188 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 1f61fd1d8bf44f64b73ade11c0224df0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:00.016036 16537 catalog_manager.cc:5526] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.022663 17324 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:00.024777 16637 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:00.024986 17636 tablet_replica.cc:331] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:00.025949 17636 raft_consensus.cc:2238] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:00.026500 17553 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:00.028126 17553 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:00.029659 17636 raft_consensus.cc:2267] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:00.032876 17636 ts_tablet_manager.cc:1905] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:00.034682 16532 catalog_manager.cc:5526] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:00.046692 16707 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:00.061952 17612 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:00.032392 16784 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.067677 16707 consensus_queue.cc:237] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:00.079813 17321 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:00.079952 16639 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:00.081542 17615 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:00.083138 17575 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:00.089618 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f61fd1d8bf44f64b73ade11c0224df0 with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:00.108788 17636 ts_tablet_manager.cc:1918] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:00.109285 17636 log.cc:1198] T 8c10f3f7e0f74c9f988fa01e7843e961 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/8c10f3f7e0f74c9f988fa01e7843e961
I20250112 20:58:00.111013 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 8c10f3f7e0f74c9f988fa01e7843e961 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:00.111222 17640 tablet_replica.cc:331] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:00.120591 17640 raft_consensus.cc:2238] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:00.121237 17640 raft_consensus.cc:2267] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:00.120707 17321 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.115429 17575 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.132715 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 4799def22f754c9881fd8245eb8f75b3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:00.143344 16632 raft_consensus.cc:2949] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.150379 17613 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.152987 17640 ts_tablet_manager.cc:1905] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:00.155000 16532 catalog_manager.cc:5526] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:00.168584 16707 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.170879 16637 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.170329 17321 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.199344 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 4799def22f754c9881fd8245eb8f75b3 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:00.199288 16537 catalog_manager.cc:5526] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.205121 17640 ts_tablet_manager.cc:1918] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:00.205688 17640 log.cc:1198] T 1f61fd1d8bf44f64b73ade11c0224df0 P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/1f61fd1d8bf44f64b73ade11c0224df0
I20250112 20:58:00.205885 17642 tablet_replica.cc:331] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:00.207785 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 1f61fd1d8bf44f64b73ade11c0224df0 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:00.206966 17642 raft_consensus.cc:2238] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:00.208981 17642 raft_consensus.cc:2267] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:00.213265 16782 consensus_queue.cc:237] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:00.216145 17642 ts_tablet_manager.cc:1905] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:00.224923 17321 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:00.224980 16714 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:00.226292 17614 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:00.227427 17613 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:58:00.236726 17614 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.239691 17642 ts_tablet_manager.cc:1918] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:00.241326 17642 log.cc:1198] T 4799def22f754c9881fd8245eb8f75b3 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/4799def22f754c9881fd8245eb8f75b3
I20250112 20:58:00.246358 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet 4799def22f754c9881fd8245eb8f75b3 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:00.247077 17321 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.249765 16707 raft_consensus.cc:2949] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.252486 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 44e603d3a09141d296df5d386a3b4d11 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:00.260747 16532 catalog_manager.cc:5526] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:00.272569 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e603d3a09141d296df5d386a3b4d11 with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:00.273492 17640 tablet_replica.cc:331] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:00.274431 17640 raft_consensus.cc:2238] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:00.274987 17640 raft_consensus.cc:2267] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:00.277920 17640 ts_tablet_manager.cc:1905] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:00.294943 17640 ts_tablet_manager.cc:1918] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:00.295235 17640 log.cc:1198] T 44e603d3a09141d296df5d386a3b4d11 P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/44e603d3a09141d296df5d386a3b4d11
I20250112 20:58:00.296706 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 44e603d3a09141d296df5d386a3b4d11 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:01.401710 16637 consensus_queue.cc:237] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:01.410674 16784 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:01.411851 17617 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:01.412571 16708 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:01.413962 17617 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:01.415084 16603 consensus_peers.cc:487] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b17fe6b5636e4cc9a2e96751cd7a14c4. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:01.419049 17580 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.420789 16784 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.422255 16708 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.432806 16637 consensus_queue.cc:237] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:01.437026 16534 catalog_manager.cc:5526] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:01.443223 16708 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:01.444998 16784 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:01.445744 17643 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:01.447516 17617 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:01.449617 16603 consensus_peers.cc:487] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ef14661bd7346f68101fe2a0fb064bb. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:01.459739 17643 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.462402 16784 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.468701 16707 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.479663 16637 consensus_queue.cc:237] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
W20250112 20:58:01.488969 16603 consensus_peers.cc:487] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82159eb8aadb4973b65d7edc4f417d84. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:01.491420 16784 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:01.492017 16707 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:01.493263 17643 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:01.494501 17580 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:01.490675 16533 catalog_manager.cc:5526] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.504861 17580 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.508944 16784 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.507696 16708 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.522367 16707 consensus_queue.cc:237] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:01.526228 16534 catalog_manager.cc:5526] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.538743 16784 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:01.539950 17575 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:01.540952 16678 consensus_peers.cc:487] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14c4d77e5fce4c0e92935eb3a168cb77. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:01.541181 16637 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:01.542451 17575 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:01.550115 17575 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.552881 16637 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.557355 16783 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:01.566926 16533 catalog_manager.cc:5526] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:01.904397 17664 ts_tablet_manager.cc:927] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:01.906309 17664 tablet_copy_client.cc:323] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:01.907745 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 7ef14661bd7346f68101fe2a0fb064bb from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:01.908285 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 7ef14661bd7346f68101fe2a0fb064bb from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-7ef14661bd7346f68101fe2a0fb064bb
I20250112 20:58:01.916939 16649 tablet_copy_source_session.cc:215] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:01.928409 17664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ef14661bd7346f68101fe2a0fb064bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:01.943217 17664 tablet_copy_client.cc:806] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:01.943743 17664 tablet_copy_client.cc:670] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:01.957401 17664 tablet_copy_client.cc:538] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:01.965133 17664 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:01.972035 17666 ts_tablet_manager.cc:927] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:01.976166 17666 tablet_copy_client.cc:323] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:01.979581 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet b17fe6b5636e4cc9a2e96751cd7a14c4 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:01.980266 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet b17fe6b5636e4cc9a2e96751cd7a14c4 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-b17fe6b5636e4cc9a2e96751cd7a14c4
I20250112 20:58:01.989884 17664 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:01.990578 17664 tablet_bootstrap.cc:492] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:01.991199 17664 ts_tablet_manager.cc:1397] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.012s
I20250112 20:58:01.993505 17664 raft_consensus.cc:357] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:01.994307 17664 raft_consensus.cc:738] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:01.994901 17664 consensus_queue.cc:260] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:01.996500 16649 tablet_copy_source_session.cc:215] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:01.998589 17664 ts_tablet_manager.cc:1428] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20250112 20:58:02.000067 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-7ef14661bd7346f68101fe2a0fb064bb received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:02.000504 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-7ef14661bd7346f68101fe2a0fb064bb on tablet 7ef14661bd7346f68101fe2a0fb064bb with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:02.007118 17666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17fe6b5636e4cc9a2e96751cd7a14c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:02.020869 17666 tablet_copy_client.cc:806] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:02.021386 17666 tablet_copy_client.cc:670] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:02.024734 17666 tablet_copy_client.cc:538] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:02.032258 17666 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:02.050503 17666 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:02.051692 17666 tablet_bootstrap.cc:492] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:02.052361 17666 ts_tablet_manager.cc:1397] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.009s
I20250112 20:58:02.054898 17666 raft_consensus.cc:357] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:02.055496 17666 raft_consensus.cc:738] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:02.055985 17666 consensus_queue.cc:260] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:02.058904 17666 ts_tablet_manager.cc:1428] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:58:02.060819 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-b17fe6b5636e4cc9a2e96751cd7a14c4 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:02.061206 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-b17fe6b5636e4cc9a2e96751cd7a14c4 on tablet b17fe6b5636e4cc9a2e96751cd7a14c4 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:02.093757 17666 ts_tablet_manager.cc:927] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:02.095147 17666 tablet_copy_client.cc:323] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:02.096405 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 82159eb8aadb4973b65d7edc4f417d84 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:02.096916 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 82159eb8aadb4973b65d7edc4f417d84 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-82159eb8aadb4973b65d7edc4f417d84
I20250112 20:58:02.102865 16649 tablet_copy_source_session.cc:215] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:02.106886 17666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82159eb8aadb4973b65d7edc4f417d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:02.118716 17666 tablet_copy_client.cc:806] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:02.119226 17666 tablet_copy_client.cc:670] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:02.122375 17666 tablet_copy_client.cc:538] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:02.129382 17666 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:02.142311 17664 ts_tablet_manager.cc:927] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:02.145331 17664 tablet_copy_client.cc:323] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:02.147189 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 14c4d77e5fce4c0e92935eb3a168cb77 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:02.147694 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 14c4d77e5fce4c0e92935eb3a168cb77 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-14c4d77e5fce4c0e92935eb3a168cb77
I20250112 20:58:02.151584 17666 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:02.153662 17666 tablet_bootstrap.cc:492] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:02.154279 17666 ts_tablet_manager.cc:1397] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.004s
I20250112 20:58:02.156960 17666 raft_consensus.cc:357] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:02.157771 17666 raft_consensus.cc:738] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:02.158375 17666 consensus_queue.cc:260] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:02.159106 16724 tablet_copy_source_session.cc:215] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:02.161907 17664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c4d77e5fce4c0e92935eb3a168cb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:02.177428 17666 ts_tablet_manager.cc:1428] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.023s user 0.007s sys 0.001s
I20250112 20:58:02.179366 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-82159eb8aadb4973b65d7edc4f417d84 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:02.179774 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-82159eb8aadb4973b65d7edc4f417d84 on tablet 82159eb8aadb4973b65d7edc4f417d84 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:02.196724 17664 tablet_copy_client.cc:806] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:02.197304 17664 tablet_copy_client.cc:670] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:02.201141 17664 tablet_copy_client.cc:538] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:02.208248 17664 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:02.231153 17664 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:02.231786 17664 tablet_bootstrap.cc:492] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:02.232610 17664 ts_tablet_manager.cc:1397] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.008s
I20250112 20:58:02.234678 17664 raft_consensus.cc:357] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:02.235512 17664 raft_consensus.cc:738] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:02.236133 17664 consensus_queue.cc:260] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:02.238543 17664 ts_tablet_manager.cc:1428] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:58:02.239785 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-14c4d77e5fce4c0e92935eb3a168cb77 received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:02.240173 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-14c4d77e5fce4c0e92935eb3a168cb77 on tablet 14c4d77e5fce4c0e92935eb3a168cb77 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:02.367147 17324 raft_consensus.cc:1212] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:02.529999 17324 raft_consensus.cc:1212] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:02.553078 17324 raft_consensus.cc:1212] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:02.645777 17324 raft_consensus.cc:1212] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:02.766971 17617 raft_consensus.cc:1059] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:02.768898 17617 consensus_queue.cc:237] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:02.774000 17324 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:02.775049 16784 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:02.776198 17617 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:02.776877 16708 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:02.777033 17643 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:02.777998 17580 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:02.790563 17643 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.793804 16708 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.795418 16784 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.803146 17324 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.811147 16532 catalog_manager.cc:5526] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.835479 16639 consensus_queue.cc:237] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:02.840167 17324 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:02.840201 16784 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:02.841624 17617 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:02.842445 17643 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:02.848580 17580 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.850062 17324 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.854693 16784 raft_consensus.cc:2949] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.862324 16534 catalog_manager.cc:5526] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.869952 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 7ef14661bd7346f68101fe2a0fb064bb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:02.876520 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef14661bd7346f68101fe2a0fb064bb with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:02.880257 17687 tablet_replica.cc:331] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:02.890452 17687 raft_consensus.cc:2238] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:02.891302 17687 raft_consensus.cc:2267] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:02.894402 17687 ts_tablet_manager.cc:1905] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:02.906435 17687 ts_tablet_manager.cc:1918] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:02.906937 17687 log.cc:1198] T 7ef14661bd7346f68101fe2a0fb064bb P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/7ef14661bd7346f68101fe2a0fb064bb
I20250112 20:58:02.907254 17643 raft_consensus.cc:1059] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:02.908923 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 7ef14661bd7346f68101fe2a0fb064bb (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:02.909521 17643 consensus_queue.cc:237] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:02.922938 16784 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:02.922971 17324 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:02.923367 16707 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:02.924360 17580 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:02.925186 17683 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:02.926062 17617 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:02.937178 17580 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.939754 17324 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.948141 16707 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.953316 16537 catalog_manager.cc:5526] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.969313 16637 consensus_queue.cc:237] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:02.977945 17321 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:02.984539 17643 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:02.986618 16709 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:02.989805 17643 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:02.994437 17683 raft_consensus.cc:1059] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:02.994968 17643 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.997656 17321 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:02.998593 17683 consensus_queue.cc:237] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:03.008184 16708 raft_consensus.cc:2949] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.024839 16784 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:03.026046 17683 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:03.026623 16708 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:03.027890 17580 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:03.033808 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet b17fe6b5636e4cc9a2e96751cd7a14c4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:03.036382 17324 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:03.041888 16537 catalog_manager.cc:5526] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.043174 17683 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:03.047410 17683 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.051726 16708 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.055107 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet b17fe6b5636e4cc9a2e96751cd7a14c4 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:03.058485 17324 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.063057 16783 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.073298 17696 tablet_replica.cc:331] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:03.074097 17696 raft_consensus.cc:2238] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:03.074548 17696 pending_rounds.cc:62] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Trying to abort 1 pending ops.
I20250112 20:58:03.074796 17696 pending_rounds.cc:69] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7113586617992421376 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } } }
I20250112 20:58:03.075699 17696 raft_consensus.cc:2911] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20250112 20:58:03.076160 17696 raft_consensus.cc:2883] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20250112 20:58:03.076557 17696 raft_consensus.cc:2267] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:03.076642 16534 catalog_manager.cc:5526] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.080132 17696 ts_tablet_manager.cc:1905] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:03.101620 16637 consensus_queue.cc:237] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:03.106825 17324 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:03.107606 16784 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:03.109140 17580 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:03.109989 17683 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:03.104771 17696 ts_tablet_manager.cc:1918] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:03.112126 17696 log.cc:1198] T b17fe6b5636e4cc9a2e96751cd7a14c4 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/b17fe6b5636e4cc9a2e96751cd7a14c4
I20250112 20:58:03.113973 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet b17fe6b5636e4cc9a2e96751cd7a14c4 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:03.115516 17580 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.117254 17324 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.120532 16783 raft_consensus.cc:2949] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.123011 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 82159eb8aadb4973b65d7edc4f417d84 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:03.131227 16534 catalog_manager.cc:5526] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.144300 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 82159eb8aadb4973b65d7edc4f417d84 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:03.145174 17687 tablet_replica.cc:331] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:03.146214 17687 raft_consensus.cc:2238] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:03.146885 17687 raft_consensus.cc:2267] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:03.149473 17687 ts_tablet_manager.cc:1905] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:03.162344 17687 ts_tablet_manager.cc:1918] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:03.162703 17687 log.cc:1198] T 82159eb8aadb4973b65d7edc4f417d84 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/82159eb8aadb4973b65d7edc4f417d84
I20250112 20:58:03.163904 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 82159eb8aadb4973b65d7edc4f417d84 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:03.224496 17676 raft_consensus.cc:1059] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:03.226873 17676 consensus_queue.cc:237] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:03.238911 17324 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:03.240497 17575 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:03.242766 16637 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:03.244141 17575 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:03.249095 16782 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:03.255227 17575 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:03.284224 17698 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.291253 16639 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.293964 17321 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.301354 16782 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.308533 16537 catalog_manager.cc:5526] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.335620 16708 consensus_queue.cc:237] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:03.339505 17321 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:03.339746 16782 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:03.340626 17698 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:03.341809 17575 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:03.347782 17698 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.349125 17321 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.351359 16782 raft_consensus.cc:2949] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.358485 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 14c4d77e5fce4c0e92935eb3a168cb77 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:03.362484 16534 catalog_manager.cc:5526] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:03.372188 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 14c4d77e5fce4c0e92935eb3a168cb77 with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:03.374147 17704 tablet_replica.cc:331] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:03.375819 17704 raft_consensus.cc:2238] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:03.376397 17704 raft_consensus.cc:2267] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:03.379376 17704 ts_tablet_manager.cc:1905] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:03.391255 17704 ts_tablet_manager.cc:1918] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:03.391539 17704 log.cc:1198] T 14c4d77e5fce4c0e92935eb3a168cb77 P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/14c4d77e5fce4c0e92935eb3a168cb77
I20250112 20:58:03.392779 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 14c4d77e5fce4c0e92935eb3a168cb77 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:04.425704 16707 consensus_queue.cc:237] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:04.431293 16637 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:04.431308 16782 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250112 20:58:04.431645 16678 consensus_peers.cc:487] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a21c93f28fc4ab3be409b95204b86dc. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:04.432927 17709 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:04.434011 17676 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:04.441119 17575 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.442356 16637 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.456290 16637 consensus_queue.cc:237] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:04.456157 16783 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.459122 16533 catalog_manager.cc:5526] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:04.485803 16707 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250112 20:58:04.486280 16603 consensus_peers.cc:487] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: abd904e2af24417f8b366444ad932e30. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:04.485839 16783 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:04.487224 17714 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:04.491142 17617 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:04.505004 17617 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.506433 16707 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.508244 16782 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.526511 16637 consensus_queue.cc:237] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:04.531816 16533 catalog_manager.cc:5526] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:58:04.537458 16603 consensus_peers.cc:487] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1773dc29c9e4d83a01c6b56f7b277d5. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:04.537731 16707 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:04.537587 16782 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:04.539685 17617 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:04.540424 17714 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:04.549288 17716 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.550351 16707 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.562811 16707 consensus_queue.cc:237] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:04.565384 16782 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.567624 16532 catalog_manager.cc:5526] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:58:04.578111 16678 consensus_peers.cc:487] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a13e4117fd64b15a4cd60943e638999. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:04.579560 16783 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:04.581005 17575 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:04.584126 16637 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:04.586670 17575 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:04.591739 17676 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.596190 16783 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.603217 16637 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:04.639070 16532 catalog_manager.cc:5526] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:04.914583 17728 ts_tablet_manager.cc:927] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:04.918164 17728 tablet_copy_client.cc:323] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:04.919931 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet abd904e2af24417f8b366444ad932e30 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:04.920540 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet abd904e2af24417f8b366444ad932e30 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-abd904e2af24417f8b366444ad932e30
I20250112 20:58:04.927793 16649 tablet_copy_source_session.cc:215] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:04.931895 17728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd904e2af24417f8b366444ad932e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:04.946952 17730 ts_tablet_manager.cc:927] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:04.949620 17728 tablet_copy_client.cc:806] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:04.950595 17728 tablet_copy_client.cc:670] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:04.953168 17732 ts_tablet_manager.cc:927] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:04.956771 17728 tablet_copy_client.cc:538] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:04.962419 17730 tablet_copy_client.cc:323] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:04.964021 17728 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:04.964068 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 9a21c93f28fc4ab3be409b95204b86dc from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:04.964709 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 9a21c93f28fc4ab3be409b95204b86dc from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-9a21c93f28fc4ab3be409b95204b86dc
I20250112 20:58:04.976949 17732 tablet_copy_client.cc:323] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:04.977464 16724 tablet_copy_source_session.cc:215] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:04.980351 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet e1773dc29c9e4d83a01c6b56f7b277d5 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:04.981894 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet e1773dc29c9e4d83a01c6b56f7b277d5 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-e1773dc29c9e4d83a01c6b56f7b277d5
I20250112 20:58:04.986451 17730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a21c93f28fc4ab3be409b95204b86dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:04.986449 17728 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:04.987579 17728 tablet_bootstrap.cc:492] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:04.988281 17728 ts_tablet_manager.cc:1397] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.018s
I20250112 20:58:04.991762 17728 raft_consensus.cc:357] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:04.992611 17728 raft_consensus.cc:738] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:04.993253 17728 consensus_queue.cc:260] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:05.006492 16649 tablet_copy_source_session.cc:215] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:05.007069 17728 ts_tablet_manager.cc:1428] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.018s user 0.007s sys 0.004s
I20250112 20:58:05.011329 17732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1773dc29c9e4d83a01c6b56f7b277d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:05.013252 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-abd904e2af24417f8b366444ad932e30 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:05.013759 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-abd904e2af24417f8b366444ad932e30 on tablet abd904e2af24417f8b366444ad932e30 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:05.025099 17730 tablet_copy_client.cc:806] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:05.025547 17730 tablet_copy_client.cc:670] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:05.030951 17730 tablet_copy_client.cc:538] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:05.034310 17732 tablet_copy_client.cc:806] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:05.034821 17732 tablet_copy_client.cc:670] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:05.038448 17732 tablet_copy_client.cc:538] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:05.040987 17730 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:05.049304 17732 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:05.051266 17728 ts_tablet_manager.cc:927] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:05.065376 17728 tablet_copy_client.cc:323] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:05.066834 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 7a13e4117fd64b15a4cd60943e638999 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:05.067325 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 7a13e4117fd64b15a4cd60943e638999 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-7a13e4117fd64b15a4cd60943e638999
I20250112 20:58:05.075747 17730 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:05.076654 17730 tablet_bootstrap.cc:492] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:05.077277 17730 ts_tablet_manager.cc:1397] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.036s user 0.016s sys 0.007s
I20250112 20:58:05.078058 16724 tablet_copy_source_session.cc:215] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:05.080026 17730 raft_consensus.cc:357] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:05.081213 17730 raft_consensus.cc:738] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:05.081880 17730 consensus_queue.cc:260] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:05.082957 17728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a13e4117fd64b15a4cd60943e638999. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:05.084487 17730 ts_tablet_manager.cc:1428] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.001s sys 0.004s
I20250112 20:58:05.088771 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-9a21c93f28fc4ab3be409b95204b86dc received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:05.089210 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-9a21c93f28fc4ab3be409b95204b86dc on tablet 9a21c93f28fc4ab3be409b95204b86dc with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:05.092913 17732 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:05.093830 17732 tablet_bootstrap.cc:492] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:05.094491 17732 ts_tablet_manager.cc:1397] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.045s user 0.020s sys 0.007s
I20250112 20:58:05.097340 17732 raft_consensus.cc:357] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:05.098196 17732 raft_consensus.cc:738] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:05.098819 17732 consensus_queue.cc:260] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:05.101238 17732 ts_tablet_manager.cc:1428] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20250112 20:58:05.102676 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-e1773dc29c9e4d83a01c6b56f7b277d5 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:05.103049 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-e1773dc29c9e4d83a01c6b56f7b277d5 on tablet e1773dc29c9e4d83a01c6b56f7b277d5 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:05.112875 17728 tablet_copy_client.cc:806] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:05.113449 17728 tablet_copy_client.cc:670] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:05.116286 17728 tablet_copy_client.cc:538] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:05.124256 17728 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:05.144083 17728 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:05.144907 17728 tablet_bootstrap.cc:492] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:05.145490 17728 ts_tablet_manager.cc:1397] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.002s
I20250112 20:58:05.148219 17728 raft_consensus.cc:357] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:05.149091 17728 raft_consensus.cc:738] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:05.149713 17728 consensus_queue.cc:260] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:05.152235 17728 ts_tablet_manager.cc:1428] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:58:05.153570 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-7a13e4117fd64b15a4cd60943e638999 received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:05.153947 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-7a13e4117fd64b15a4cd60943e638999 on tablet 7a13e4117fd64b15a4cd60943e638999 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:05.307005 17321 raft_consensus.cc:1212] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:05.324967 17321 raft_consensus.cc:1212] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:05.477418 17321 raft_consensus.cc:1212] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:05.515017 17321 raft_consensus.cc:1212] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:05.853312 17714 raft_consensus.cc:1059] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:05.855820 17714 consensus_queue.cc:237] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:05.863147 17321 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:05.866179 16707 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:05.867152 17716 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:05.869865 17743 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:05.875952 16784 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:05.877728 17716 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:58:05.891247 17617 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:05.900532 16707 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:05.908310 17321 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:05.914386 16784 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:05.919790 16533 catalog_manager.cc:5526] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:05.940372 17676 raft_consensus.cc:1059] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:05.942772 17676 consensus_queue.cc:237] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:05.962286 16632 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:05.964668 17324 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:05.966189 17740 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:05.968909 17676 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:05.985321 16784 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:05.987056 17676 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:05.998121 17740 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:05.999723 17324 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.001516 16784 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.011237 16637 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.024631 16637 consensus_queue.cc:237] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:06.031047 17321 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:06.032208 16784 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:06.033754 17716 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:06.038411 16532 catalog_manager.cc:5526] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.042057 17743 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:06.043773 17743 raft_consensus.cc:1059] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:06.043964 17617 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.046928 17743 consensus_queue.cc:237] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:06.050083 16784 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.051550 17321 raft_consensus.cc:2949] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.063306 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet abd904e2af24417f8b366444ad932e30 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:06.071480 16708 consensus_queue.cc:237] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:06.078685 16533 catalog_manager.cc:5526] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.080011 17321 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:06.080199 16707 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:06.082159 16784 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:06.082844 17740 raft_consensus.cc:1059] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:06.086328 17714 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:06.087994 17714 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:06.088428 16637 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:06.083603 17324 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:06.088076 17740 consensus_queue.cc:237] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:06.090057 17709 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:06.091584 17676 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:06.097311 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet abd904e2af24417f8b366444ad932e30 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:06.104372 17617 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:06.107777 17709 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.115409 17321 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.118499 16784 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:06.119071 17759 tablet_replica.cc:331] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:06.120196 17759 raft_consensus.cc:2238] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:06.120855 17759 raft_consensus.cc:2267] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:06.121354 17760 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:06.123278 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 9a21c93f28fc4ab3be409b95204b86dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:06.123960 17324 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:06.128630 16639 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:06.130892 17760 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:06.129493 16637 raft_consensus.cc:2949] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.132234 16537 catalog_manager.cc:5526] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:06.134948 17759 ts_tablet_manager.cc:1905] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:06.157030 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a21c93f28fc4ab3be409b95204b86dc with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:06.151471 17709 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.154852 17743 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.162418 17760 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:06.162514 17322 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.153276 17324 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.170099 17764 tablet_replica.cc:331] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:06.171806 17759 ts_tablet_manager.cc:1918] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:06.172209 17759 log.cc:1198] T abd904e2af24417f8b366444ad932e30 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/abd904e2af24417f8b366444ad932e30
I20250112 20:58:06.180377 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet abd904e2af24417f8b366444ad932e30 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:06.181043 16785 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.186851 17764 raft_consensus.cc:2238] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:06.189189 17764 raft_consensus.cc:2267] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:06.193125 16708 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.201263 17764 ts_tablet_manager.cc:1905] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:06.205783 16537 catalog_manager.cc:5526] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:06.207482 16784 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.213706 16534 catalog_manager.cc:5526] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:06.215545 16639 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.238657 16708 consensus_queue.cc:237] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:06.255045 17324 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:06.256163 17740 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:06.262065 17709 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.261608 16639 consensus_queue.cc:237] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:06.266258 16785 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:06.268832 17740 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:06.272831 16708 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:06.274199 17322 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:06.274024 17743 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:06.275625 17714 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:06.277091 16785 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.282912 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 7a13e4117fd64b15a4cd60943e638999 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:06.283771 17764 ts_tablet_manager.cc:1918] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:06.284226 17764 log.cc:1198] T 9a21c93f28fc4ab3be409b95204b86dc P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/9a21c93f28fc4ab3be409b95204b86dc
I20250112 20:58:06.286309 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet 9a21c93f28fc4ab3be409b95204b86dc (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:06.290716 17324 raft_consensus.cc:2949] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.296330 17617 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.302507 16532 catalog_manager.cc:5526] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:06.305670 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet e1773dc29c9e4d83a01c6b56f7b277d5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:06.310220 17322 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.316877 16708 raft_consensus.cc:2949] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:06.319998 16534 catalog_manager.cc:5526] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:06.331396 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a13e4117fd64b15a4cd60943e638999 with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:06.334111 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet e1773dc29c9e4d83a01c6b56f7b277d5 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:06.335110 17764 tablet_replica.cc:331] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:06.336123 17764 raft_consensus.cc:2238] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:06.336890 17764 raft_consensus.cc:2267] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:06.340866 17764 ts_tablet_manager.cc:1905] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:06.346702 17767 tablet_replica.cc:331] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:06.347760 17767 raft_consensus.cc:2238] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:06.348563 17767 raft_consensus.cc:2267] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:06.351576 17767 ts_tablet_manager.cc:1905] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:06.356042 17764 ts_tablet_manager.cc:1918] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:06.356485 17764 log.cc:1198] T e1773dc29c9e4d83a01c6b56f7b277d5 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/e1773dc29c9e4d83a01c6b56f7b277d5
I20250112 20:58:06.358120 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet e1773dc29c9e4d83a01c6b56f7b277d5 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:06.367769 17767 ts_tablet_manager.cc:1918] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:06.368167 17767 log.cc:1198] T 7a13e4117fd64b15a4cd60943e638999 P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/7a13e4117fd64b15a4cd60943e638999
I20250112 20:58:06.369482 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 7a13e4117fd64b15a4cd60943e638999 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:06.815410 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:06.859086 16707 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:06.859710 16707 raft_consensus.cc:604] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:06.861743 16707 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:06.862139 16707 raft_consensus.cc:604] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:06.871984 16785 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:06.872478 16785 raft_consensus.cc:604] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:06.882329 16639 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "abd904e2af24417f8b366444ad932e30"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:06.882874 16639 raft_consensus.cc:604] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:06.885340 16639 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:06.885841 16639 raft_consensus.cc:604] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:06.888918 16708 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:06.889416 16708 raft_consensus.cc:604] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:06.890409 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20250112 20:58:06.891893 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:06.892503 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:06.974898 17743 raft_consensus.cc:988] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:06.975430 17743 raft_consensus.cc:1076] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:06.976779 17324 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "abd904e2af24417f8b366444ad932e30"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53250
I20250112 20:58:06.977280 17324 raft_consensus.cc:491] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:06.977545 17324 raft_consensus.cc:3054] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:06.982450 17324 raft_consensus.cc:513] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:06.984110 17324 leader_election.cc:290] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:06.985117 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abd904e2af24417f8b366444ad932e30" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:06.985136 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abd904e2af24417f8b366444ad932e30" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:06.985795 16785 raft_consensus.cc:3054] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:06.985867 16639 raft_consensus.cc:3049] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:06.986260 16639 raft_consensus.cc:738] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:06.987556 16639 consensus_queue.cc:260] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:06.988675 16639 raft_consensus.cc:3054] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:06.990329 16785 raft_consensus.cc:2463] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:06.991299 17290 leader_election.cc:304] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:06.992874 17772 raft_consensus.cc:2798] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:06.993445 16639 raft_consensus.cc:2463] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:06.995157 17772 raft_consensus.cc:695] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:06.996898 17772 consensus_queue.cc:237] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.000648 17709 raft_consensus.cc:988] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.001037 17709 raft_consensus.cc:1076] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.004724 16532 catalog_manager.cc:5526] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:07.008481 17324 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53256
I20250112 20:58:07.009027 17324 raft_consensus.cc:491] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:07.009361 17324 raft_consensus.cc:3054] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.015439 17324 raft_consensus.cc:513] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.017575 17324 leader_election.cc:290] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:07.017894 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:07.018035 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd7c3a6eea9043f4b438c4e28aca966e" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:07.018430 16707 raft_consensus.cc:3049] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:07.018638 16785 raft_consensus.cc:3054] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.018785 16707 raft_consensus.cc:738] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:07.019538 16707 consensus_queue.cc:260] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.020779 16707 raft_consensus.cc:3054] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.023844 16785 raft_consensus.cc:2463] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.024710 17290 leader_election.cc:304] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:07.025313 17772 raft_consensus.cc:2798] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:07.025759 17772 raft_consensus.cc:695] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:07.026613 17772 consensus_queue.cc:237] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.026084 16707 raft_consensus.cc:2463] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.040557 17709 raft_consensus.cc:988] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.041005 17709 raft_consensus.cc:1076] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.042245 17324 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53256
I20250112 20:58:07.042697 17324 raft_consensus.cc:491] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:07.042982 17324 raft_consensus.cc:3054] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.047260 16532 catalog_manager.cc:5526] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:07.048703 17324 raft_consensus.cc:513] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.052619 17324 leader_election.cc:290] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:07.053757 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:07.054127 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14c4d77e5fce4c0e92935eb3a168cb77" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:07.054248 16707 raft_consensus.cc:3049] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:07.054673 16785 raft_consensus.cc:3054] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.054652 16707 raft_consensus.cc:738] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:07.056504 16707 consensus_queue.cc:260] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.057654 16707 raft_consensus.cc:3054] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.060926 16785 raft_consensus.cc:2463] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.061749 17290 leader_election.cc:304] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:07.062330 17772 raft_consensus.cc:2798] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:07.062781 17772 raft_consensus.cc:695] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:07.063256 16707 raft_consensus.cc:2463] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.063772 17772 consensus_queue.cc:237] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.070452 16532 catalog_manager.cc:5526] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:07.104132 17766 raft_consensus.cc:988] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.104635 17743 raft_consensus.cc:1076] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.106029 17324 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53250
I20250112 20:58:07.106482 17324 raft_consensus.cc:491] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:07.106801 17324 raft_consensus.cc:3054] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.110972 17324 raft_consensus.cc:513] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.112097 17324 leader_election.cc:290] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:07.112792 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:07.112911 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b17fe6b5636e4cc9a2e96751cd7a14c4" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:07.113301 16637 raft_consensus.cc:3049] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:07.113444 16707 raft_consensus.cc:3054] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.113569 16637 raft_consensus.cc:738] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:07.114270 16637 consensus_queue.cc:260] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.115253 16637 raft_consensus.cc:3054] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.117813 16707 raft_consensus.cc:2463] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.118781 17287 leader_election.cc:304] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters:
I20250112 20:58:07.119393 17772 raft_consensus.cc:2798] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:07.119841 17772 raft_consensus.cc:695] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:07.120360 16637 raft_consensus.cc:2463] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.120656 17772 consensus_queue.cc:237] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.128763 16532 catalog_manager.cc:5526] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:07.217600 17740 raft_consensus.cc:988] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.218015 17709 raft_consensus.cc:1076] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.219110 17322 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53256
I20250112 20:58:07.219547 17322 raft_consensus.cc:491] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:07.219877 17322 raft_consensus.cc:3054] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.222409 17741 raft_consensus.cc:988] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.222810 17674 raft_consensus.cc:1076] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:07.224120 17324 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53246
I20250112 20:58:07.224556 17324 raft_consensus.cc:491] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:07.224804 17324 raft_consensus.cc:3054] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.225525 17322 raft_consensus.cc:513] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.226783 17322 leader_election.cc:290] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:07.227418 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:07.227613 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f63edc7cd344fc88b39d2a6b2fbceb" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:07.228013 16707 raft_consensus.cc:3049] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:07.228147 16785 raft_consensus.cc:3054] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.228363 16707 raft_consensus.cc:738] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:07.229499 16707 consensus_queue.cc:260] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.229951 17324 raft_consensus.cc:513] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.230666 16707 raft_consensus.cc:3054] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.233157 16708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:07.233326 17324 leader_election.cc:290] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:07.233304 16785 raft_consensus.cc:2463] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.233181 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e603d3a09141d296df5d386a3b4d11" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:07.234432 16784 raft_consensus.cc:3049] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:07.234706 16784 raft_consensus.cc:738] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:07.235352 16784 consensus_queue.cc:260] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.236231 16784 raft_consensus.cc:3054] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.233755 16708 raft_consensus.cc:3054] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:07.240772 16784 raft_consensus.cc:2463] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.241464 16708 raft_consensus.cc:2463] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.234509 17290 leader_election.cc:304] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:07.242164 17287 leader_election.cc:304] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters:
I20250112 20:58:07.242363 17772 raft_consensus.cc:2798] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:07.242995 17773 raft_consensus.cc:2798] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:07.245677 17772 raft_consensus.cc:695] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:07.246832 17772 consensus_queue.cc:237] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.247936 16707 raft_consensus.cc:2463] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:07.248804 17773 raft_consensus.cc:695] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:07.249668 17773 consensus_queue.cc:237] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:07.258833 16532 catalog_manager.cc:5526] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:07.260309 16532 catalog_manager.cc:5526] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:07.374678 16785 consensus_queue.cc:237] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
W20250112 20:58:07.380839 16752 consensus_peers.cc:487] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b3eb6b1d35d4c23bb0732c8c18c56ab. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:07.380926 16639 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.382375 16639 raft_consensus.cc:1270] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:07.383001 16708 raft_consensus.cc:1270] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:07.382570 17773 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250112 20:58:07.383642 17674 consensus_queue.cc:1035] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:07.384490 17741 consensus_queue.cc:1035] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:07.393004 17674 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.397840 16637 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.410321 16784 consensus_queue.cc:237] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:07.410504 16711 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.415468 16532 catalog_manager.cc:5526] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:07.416375 16783 raft_consensus.cc:1270] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.425503 17772 consensus_queue.cc:1035] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 20:58:07.438126 16752 consensus_peers.cc:487] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 305481b596b746b1a03df8d76e4057fb. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:07.439497 16637 raft_consensus.cc:1270] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:07.440538 16714 raft_consensus.cc:1270] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:07.440889 17741 consensus_queue.cc:1035] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:07.443236 17785 consensus_queue.cc:1035] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:07.453092 16714 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.456197 17772 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250112 20:58:07.490196 17741 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.492929 16639 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.498203 16784 raft_consensus.cc:1270] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.499856 17780 consensus_queue.cc:1035] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250112 20:58:07.505394 16707 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.520303 16639 consensus_queue.cc:237] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:07.522404 16532 catalog_manager.cc:5526] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.523113 16784 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.525743 17780 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250112 20:58:07.527988 16603 consensus_peers.cc:487] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a5ffa8e639d640a3b7f19d9857148f40. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:07.528335 16783 raft_consensus.cc:1270] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:07.529702 16711 raft_consensus.cc:1270] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:07.529923 17714 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:07.537009 17766 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:07.537865 16711 raft_consensus.cc:1270] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.544911 17780 consensus_queue.cc:1035] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:07.551398 17766 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.559135 16711 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.586954 16639 consensus_queue.cc:237] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:07.594225 16537 catalog_manager.cc:5526] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:07.597858 16783 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:07.599135 17766 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:07.601295 16603 consensus_peers.cc:487] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e23f9778f8494e11b9d942035720e363. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:07.601337 16711 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:07.602936 17766 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:07.613445 16784 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.617894 17807 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.623175 16785 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.630941 16707 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.633111 17780 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:07.650069 16537 catalog_manager.cc:5526] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.654662 16707 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:07.674664 16639 raft_consensus.cc:1270] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.682915 17781 consensus_queue.cc:1035] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:07.696583 16707 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.698360 17780 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:07.715188 16785 raft_consensus.cc:1270] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.716409 17780 consensus_queue.cc:1035] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:07.745983 16707 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.747010 17780 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:07.757895 16785 raft_consensus.cc:1270] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:07.759173 17773 consensus_queue.cc:1035] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:07.835179 17818 ts_tablet_manager.cc:927] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:07.845412 17818 tablet_copy_client.cc:323] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:58:07.849678 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:58:07.850241 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-9b3eb6b1d35d4c23bb0732c8c18c56ab
I20250112 20:58:07.860042 16798 tablet_copy_source_session.cc:215] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:07.863569 17820 ts_tablet_manager.cc:927] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:07.863620 17818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:07.873415 17820 tablet_copy_client.cc:323] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:58:07.874908 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet 305481b596b746b1a03df8d76e4057fb from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:58:07.875407 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet 305481b596b746b1a03df8d76e4057fb from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-305481b596b746b1a03df8d76e4057fb
I20250112 20:58:07.883210 16798 tablet_copy_source_session.cc:215] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:07.884528 17818 tablet_copy_client.cc:806] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:07.885113 17818 tablet_copy_client.cc:670] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:07.886569 17820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 305481b596b746b1a03df8d76e4057fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:07.889683 17818 tablet_copy_client.cc:538] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:07.897828 17818 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:07.900718 17820 tablet_copy_client.cc:806] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:07.901384 17820 tablet_copy_client.cc:670] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:07.904683 17820 tablet_copy_client.cc:538] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:07.912520 17820 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:07.916359 17818 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:07.917224 17818 tablet_bootstrap.cc:492] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:07.917793 17818 ts_tablet_manager.cc:1397] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.001s
I20250112 20:58:07.920353 17818 raft_consensus.cc:357] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:07.921041 17818 raft_consensus.cc:738] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:07.921494 17818 consensus_queue.cc:260] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:07.923713 17818 ts_tablet_manager.cc:1428] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20250112 20:58:07.924223 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:07.928987 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-9b3eb6b1d35d4c23bb0732c8c18c56ab received from {username='slave'} at 127.0.0.1:51714
I20250112 20:58:07.929522 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-9b3eb6b1d35d4c23bb0732c8c18c56ab on tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:07.936388 17820 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:07.937288 17820 tablet_bootstrap.cc:492] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:07.938025 17820 ts_tablet_manager.cc:1397] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.026s user 0.018s sys 0.002s
I20250112 20:58:07.942379 17820 raft_consensus.cc:357] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:07.943203 17820 raft_consensus.cc:738] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:07.943760 17820 consensus_queue.cc:260] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:07.948457 17820 ts_tablet_manager.cc:1428] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20250112 20:58:07.950066 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-305481b596b746b1a03df8d76e4057fb received from {username='slave'} at 127.0.0.1:51714
I20250112 20:58:07.950452 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-305481b596b746b1a03df8d76e4057fb on tablet 305481b596b746b1a03df8d76e4057fb with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:07.980098 16632 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:07.980664 16632 raft_consensus.cc:604] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:07.982648 16784 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:07.983104 16784 raft_consensus.cc:604] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:07.985147 16632 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8c10f3f7e0f74c9f988fa01e7843e961"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:07.985636 16632 raft_consensus.cc:604] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:07.987491 16634 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "df9c04a57cac449e8efae9c7689ad690"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:07.988016 16634 raft_consensus.cc:604] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:07.990120 16634 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e1773dc29c9e4d83a01c6b56f7b277d5"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:07.990494 16634 raft_consensus.cc:604] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:07.991247 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20250112 20:58:07.992621 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:07.993185 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:08.024950 17674 raft_consensus.cc:988] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:08.025353 17674 raft_consensus.cc:1076] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:08.026696 16632 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50052
I20250112 20:58:08.027177 16632 raft_consensus.cc:491] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:08.027464 16632 raft_consensus.cc:3054] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.034444 16632 raft_consensus.cc:513] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.036883 16708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:08.037032 16632 leader_election.cc:290] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:08.037403 16708 raft_consensus.cc:3054] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.042646 16708 raft_consensus.cc:2463] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:08.054049 16602 leader_election.cc:304] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:08.054721 17807 raft_consensus.cc:2798] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:08.055164 17807 raft_consensus.cc:695] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:08.055907 17820 ts_tablet_manager.cc:927] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:08.056109 17807 consensus_queue.cc:237] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.044876 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:08.058519 16785 raft_consensus.cc:3049] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:08.058804 16785 raft_consensus.cc:738] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:08.059502 16785 consensus_queue.cc:260] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.061829 16785 raft_consensus.cc:3054] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.072628 16534 catalog_manager.cc:5526] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:08.074532 16785 raft_consensus.cc:2463] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:08.077972 17820 tablet_copy_client.cc:323] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:08.080684 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet a5ffa8e639d640a3b7f19d9857148f40 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:08.081185 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet a5ffa8e639d640a3b7f19d9857148f40 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-a5ffa8e639d640a3b7f19d9857148f40
I20250112 20:58:08.089496 16649 tablet_copy_source_session.cc:215] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:08.092139 17820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ffa8e639d640a3b7f19d9857148f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:08.107345 17820 tablet_copy_client.cc:806] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:08.107877 17820 tablet_copy_client.cc:670] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:08.122148 17807 raft_consensus.cc:988] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:08.126817 17807 raft_consensus.cc:1076] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:08.128253 16708 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:59006
I20250112 20:58:08.128758 16708 raft_consensus.cc:491] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:08.128990 16708 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.132556 17820 tablet_copy_client.cc:538] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:08.134538 16708 raft_consensus.cc:513] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.137311 16632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:08.137849 16632 raft_consensus.cc:3049] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:08.138130 16632 raft_consensus.cc:738] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:08.138731 16632 consensus_queue.cc:260] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.139464 16708 leader_election.cc:290] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:08.139729 16632 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.140182 17820 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:08.144496 17807 raft_consensus.cc:988] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:08.144914 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:08.145102 17811 raft_consensus.cc:1076] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:08.145432 16784 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.149237 17322 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e1773dc29c9e4d83a01c6b56f7b277d5"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53250
I20250112 20:58:08.149775 17322 raft_consensus.cc:491] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:08.150135 17322 raft_consensus.cc:3054] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.150504 16632 raft_consensus.cc:2463] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:08.151131 16784 raft_consensus.cc:2463] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:08.151640 16678 leader_election.cc:304] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:08.152347 17709 raft_consensus.cc:2798] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:08.152818 17709 raft_consensus.cc:695] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:08.153476 17709 consensus_queue.cc:237] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.175299 17322 raft_consensus.cc:513] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.181349 16532 catalog_manager.cc:5526] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:08.185689 16708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1773dc29c9e4d83a01c6b56f7b277d5" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:08.186235 16708 raft_consensus.cc:3054] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.189286 16632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1773dc29c9e4d83a01c6b56f7b277d5" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:08.189919 16632 raft_consensus.cc:3049] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:08.190260 16632 raft_consensus.cc:738] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:08.191336 17322 leader_election.cc:290] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:08.190912 16632 consensus_queue.cc:260] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.193658 16632 raft_consensus.cc:3054] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.197290 16708 raft_consensus.cc:2463] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:08.198190 17287 leader_election.cc:304] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters:
I20250112 20:58:08.198961 17773 raft_consensus.cc:2798] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:08.199327 16632 raft_consensus.cc:2463] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:08.199431 17773 raft_consensus.cc:695] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:08.200259 17773 consensus_queue.cc:237] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.202689 17818 ts_tablet_manager.cc:927] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:08.204250 17818 tablet_copy_client.cc:323] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:08.206036 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet e23f9778f8494e11b9d942035720e363 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:08.206692 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet e23f9778f8494e11b9d942035720e363 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-e23f9778f8494e11b9d942035720e363
I20250112 20:58:08.210386 17820 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:08.211251 17820 tablet_bootstrap.cc:492] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:08.212709 17820 ts_tablet_manager.cc:1397] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.073s user 0.029s sys 0.021s
I20250112 20:58:08.215247 17820 raft_consensus.cc:357] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:08.216070 17820 raft_consensus.cc:738] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:08.216674 17820 consensus_queue.cc:260] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:08.217109 16649 tablet_copy_source_session.cc:215] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:08.220144 17820 ts_tablet_manager.cc:1428] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20250112 20:58:08.221855 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-a5ffa8e639d640a3b7f19d9857148f40 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:08.222307 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-a5ffa8e639d640a3b7f19d9857148f40 on tablet a5ffa8e639d640a3b7f19d9857148f40 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:08.222357 17818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e23f9778f8494e11b9d942035720e363. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:08.238631 16532 catalog_manager.cc:5526] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:08.244832 17818 tablet_copy_client.cc:806] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:08.245379 17818 tablet_copy_client.cc:670] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:08.248378 17818 tablet_copy_client.cc:538] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:08.256079 17818 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:08.279808 17818 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:08.280552 17818 tablet_bootstrap.cc:492] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:08.281122 17818 ts_tablet_manager.cc:1397] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.025s user 0.016s sys 0.011s
I20250112 20:58:08.283483 17818 raft_consensus.cc:357] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:08.284063 17818 raft_consensus.cc:738] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:08.284576 17818 consensus_queue.cc:260] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:08.286484 17818 ts_tablet_manager.cc:1428] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20250112 20:58:08.288452 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-e23f9778f8494e11b9d942035720e363 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:08.288872 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-e23f9778f8494e11b9d942035720e363 on tablet e23f9778f8494e11b9d942035720e363 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:08.317334 17811 raft_consensus.cc:988] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:08.317791 17811 raft_consensus.cc:1076] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:08.319051 16708 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "df9c04a57cac449e8efae9c7689ad690"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:59006
I20250112 20:58:08.319471 16708 raft_consensus.cc:491] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:08.319794 16708 raft_consensus.cc:3054] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.325039 16708 raft_consensus.cc:513] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.326702 16708 leader_election.cc:290] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:08.326936 16637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9c04a57cac449e8efae9c7689ad690" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:08.327483 16637 raft_consensus.cc:3049] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:08.327757 16637 raft_consensus.cc:738] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:08.328418 16637 consensus_queue.cc:260] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.329103 16784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df9c04a57cac449e8efae9c7689ad690" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:08.329463 16637 raft_consensus.cc:3054] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.329543 16784 raft_consensus.cc:3054] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.334369 16637 raft_consensus.cc:2463] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:08.335628 16678 leader_election.cc:304] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:08.336326 17812 raft_consensus.cc:2798] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:08.334401 16784 raft_consensus.cc:2463] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:08.336777 17812 raft_consensus.cc:695] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:08.337581 17812 consensus_queue.cc:237] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:08.349984 17324 raft_consensus.cc:1212] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:08.359005 16532 catalog_manager.cc:5526] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:08.428279 17811 raft_consensus.cc:988] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:08.428725 17809 raft_consensus.cc:1076] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:08.430128 17324 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8c10f3f7e0f74c9f988fa01e7843e961"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53250
I20250112 20:58:08.430562 17324 raft_consensus.cc:491] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:08.430852 17324 raft_consensus.cc:3054] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.435221 17324 raft_consensus.cc:513] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.437213 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10f3f7e0f74c9f988fa01e7843e961" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:08.436455 17324 leader_election.cc:290] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:08.437909 16783 raft_consensus.cc:3054] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.437222 16632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10f3f7e0f74c9f988fa01e7843e961" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:08.440111 16632 raft_consensus.cc:3049] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:08.441100 16632 raft_consensus.cc:738] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:08.441843 16632 consensus_queue.cc:260] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.443822 16783 raft_consensus.cc:2463] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:08.444907 17290 leader_election.cc:304] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:08.445528 16632 raft_consensus.cc:3054] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:08.445652 17773 raft_consensus.cc:2798] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:08.446178 17773 raft_consensus.cc:695] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:08.446959 17773 consensus_queue.cc:237] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.452237 17324 raft_consensus.cc:1212] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:08.455768 16632 raft_consensus.cc:2463] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:08.456825 16532 catalog_manager.cc:5526] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:08.579984 17322 raft_consensus.cc:1212] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:08.587952 16708 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:08.589077 17773 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:08.604035 16632 raft_consensus.cc:1270] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:08.605885 17781 consensus_queue.cc:1035] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:08.629765 16785 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:08.630585 16783 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:08.632012 17709 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:08.632947 16708 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:08.634006 17743 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:08.636901 17743 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:08.678297 17322 raft_consensus.cc:1212] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:08.691704 16632 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:08.751458 17812 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:08.753037 16678 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:41463, user_credentials={real_user=slave}} blocked reactor thread for 54896.9us
I20250112 20:58:08.769567 16639 raft_consensus.cc:1270] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:08.787535 17812 consensus_queue.cc:1035] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:08.800016 16783 raft_consensus.cc:1270] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:08.804086 17812 consensus_queue.cc:1035] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:08.826647 16632 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:08.827755 17781 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:08.854915 16783 raft_consensus.cc:1270] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:08.857856 17781 consensus_queue.cc:1035] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:08.878561 17674 raft_consensus.cc:1059] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:08.880744 17674 consensus_queue.cc:237] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.886407 16708 raft_consensus.cc:1270] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:08.888633 17322 raft_consensus.cc:1270] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:08.889762 17843 consensus_queue.cc:1035] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:08.890332 16632 raft_consensus.cc:1270] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:08.891311 17674 consensus_queue.cc:1035] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:08.893117 17674 consensus_queue.cc:1035] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:08.900815 17674 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:08.901585 17322 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:08.908043 16708 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:08.920121 16632 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:08.914557 16534 catalog_manager.cc:5526] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:08.963546 16785 consensus_queue.cc:237] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.968642 16632 raft_consensus.cc:1270] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:08.969758 17741 consensus_queue.cc:1035] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:08.970985 17322 raft_consensus.cc:1270] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:08.972239 17741 consensus_queue.cc:1035] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:08.975513 17674 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:08.980191 17322 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:08.982609 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:08.981675 16632 raft_consensus.cc:2949] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:08.987748 17809 raft_consensus.cc:1059] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:08.989871 17809 consensus_queue.cc:237] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:08.993386 16534 catalog_manager.cc:5526] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:09.003710 16785 raft_consensus.cc:1270] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.004962 16708 raft_consensus.cc:1270] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.006737 17743 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:09.008450 17743 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:09.012987 17741 raft_consensus.cc:1059] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:09.014948 17741 consensus_queue.cc:237] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.020422 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:09.023859 17322 raft_consensus.cc:1270] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.027277 17809 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:58:09.029776 17324 raft_consensus.cc:1270] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.031373 17674 consensus_queue.cc:1035] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:09.038851 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:09.040970 16707 raft_consensus.cc:1270] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.043412 17792 consensus_queue.cc:1035] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:09.058983 16639 raft_consensus.cc:1270] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.061092 17741 consensus_queue.cc:1035] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:09.063756 17809 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.074158 17857 tablet_replica.cc:331] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:09.067399 17322 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.087456 17857 raft_consensus.cc:2238] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:09.089730 17857 raft_consensus.cc:2267] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:09.093827 17743 raft_consensus.cc:1059] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:09.095511 17857 ts_tablet_manager.cc:1905] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:09.103005 17743 consensus_queue.cc:237] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.108276 17674 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.109381 17324 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.113004 16783 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.116158 16534 catalog_manager.cc:5526] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.134541 16534 catalog_manager.cc:5526] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.135871 16639 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.146281 16707 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.151012 16639 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:09.159015 16783 consensus_queue.cc:237] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.142457 16708 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.172731 17322 raft_consensus.cc:1270] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:09.173944 17741 consensus_queue.cc:1035] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:09.181823 17324 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.182612 16783 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.184963 17743 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:09.185144 16714 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:09.187397 17811 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:09.188998 17741 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.192844 17862 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:09.201521 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 305481b596b746b1a03df8d76e4057fb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:09.229980 16711 raft_consensus.cc:1270] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:09.230136 17324 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.235733 17741 consensus_queue.cc:1035] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:09.239436 16711 raft_consensus.cc:2949] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.232550 17857 ts_tablet_manager.cc:1918] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:09.247565 17811 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.249449 17857 log.cc:1198] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/9b3eb6b1d35d4c23bb0732c8c18c56ab
I20250112 20:58:09.218986 16639 raft_consensus.cc:604] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:09.257032 16639 consensus_queue.cc:237] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.260317 16711 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.259246 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 9b3eb6b1d35d4c23bb0732c8c18c56ab (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:09.260558 16783 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:09.262202 16783 raft_consensus.cc:604] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:09.270195 17321 raft_consensus.cc:1270] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:09.270740 17324 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.271571 16709 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:09.272197 16709 raft_consensus.cc:604] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:09.275694 17864 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:58:09.282147 16709 raft_consensus.cc:1270] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:09.285987 17862 consensus_queue.cc:1035] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:09.283788 17809 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.289831 16711 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.272411 16533 catalog_manager.cc:5526] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.295392 16532 catalog_manager.cc:5526] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:09.295324 17321 raft_consensus.cc:2949] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.299860 16782 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "963bd285f345421581941f10dbe8b90c"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:09.299451 16785 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.300431 16782 raft_consensus.cc:604] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:09.308317 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet a5ffa8e639d640a3b7f19d9857148f40 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:09.309115 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 305481b596b746b1a03df8d76e4057fb with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:09.314960 17869 tablet_replica.cc:331] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:09.316129 17869 raft_consensus.cc:2238] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:09.316799 17869 raft_consensus.cc:2267] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:09.329581 17869 ts_tablet_manager.cc:1905] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:09.328830 16708 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:09.330912 16708 raft_consensus.cc:604] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:09.330930 16534 catalog_manager.cc:5526] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.332099 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20250112 20:58:09.333549 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:09.333945 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:09.335877 16636 consensus_queue.cc:237] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.344650 17321 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:09.344805 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet a5ffa8e639d640a3b7f19d9857148f40 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:09.346875 17811 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:09.355854 16711 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:09.359827 17870 tablet_replica.cc:331] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:09.360890 17870 raft_consensus.cc:2238] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:09.361609 17870 raft_consensus.cc:2267] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:09.367270 17870 ts_tablet_manager.cc:1905] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:09.368184 17811 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:09.376346 17743 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.386478 17870 ts_tablet_manager.cc:1918] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:09.386976 17870 log.cc:1198] T a5ffa8e639d640a3b7f19d9857148f40 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/a5ffa8e639d640a3b7f19d9857148f40
I20250112 20:58:09.394774 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet e23f9778f8494e11b9d942035720e363 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:09.397559 17324 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.399343 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet a5ffa8e639d640a3b7f19d9857148f40 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:09.401005 17869 ts_tablet_manager.cc:1918] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:09.401458 17869 log.cc:1198] T 305481b596b746b1a03df8d76e4057fb P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/305481b596b746b1a03df8d76e4057fb
I20250112 20:58:09.403019 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 305481b596b746b1a03df8d76e4057fb (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:09.402956 16534 catalog_manager.cc:5526] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:09.414273 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet e23f9778f8494e11b9d942035720e363 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:09.415176 17870 tablet_replica.cc:331] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:09.416419 17870 raft_consensus.cc:2238] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:09.412583 16708 raft_consensus.cc:2949] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:09.417044 17870 raft_consensus.cc:2267] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:09.421161 17870 ts_tablet_manager.cc:1905] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:09.424976 17674 raft_consensus.cc:988] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:09.425408 17674 raft_consensus.cc:1076] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:09.426808 16711 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:58990
I20250112 20:58:09.427275 16711 raft_consensus.cc:491] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:09.427587 16711 raft_consensus.cc:3054] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.431860 17809 raft_consensus.cc:988] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:09.432421 17809 raft_consensus.cc:1076] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:09.433745 16707 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:59006
I20250112 20:58:09.434196 16707 raft_consensus.cc:491] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:09.434497 16707 raft_consensus.cc:3054] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.435896 17870 ts_tablet_manager.cc:1918] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:09.436393 17870 log.cc:1198] T e23f9778f8494e11b9d942035720e363 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/e23f9778f8494e11b9d942035720e363
I20250112 20:58:09.437839 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet e23f9778f8494e11b9d942035720e363 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:09.439375 16711 raft_consensus.cc:513] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.441738 16632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:09.441915 17709 raft_consensus.cc:988] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:09.442296 16632 raft_consensus.cc:3054] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.442391 17709 raft_consensus.cc:1076] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:09.443679 17321 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53256
I20250112 20:58:09.444154 17321 raft_consensus.cc:491] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:09.444450 17321 raft_consensus.cc:3054] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.444759 16711 leader_election.cc:290] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:09.446313 16707 raft_consensus.cc:513] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.447497 16707 leader_election.cc:290] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:09.447687 16632 raft_consensus.cc:2463] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:09.448103 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:09.448635 16639 raft_consensus.cc:3049] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:09.448642 16678 leader_election.cc:304] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:09.449026 16639 raft_consensus.cc:738] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:09.449491 17812 raft_consensus.cc:2798] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:09.449832 17812 raft_consensus.cc:695] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:09.449685 16639 consensus_queue.cc:260] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.450645 16639 raft_consensus.cc:3054] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.450551 17812 consensus_queue.cc:237] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.453963 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1696e724d24e4ebbb185476ec5ff8c09" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:09.454365 16783 raft_consensus.cc:3049] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:09.454273 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b0ecdb5a6684883976581f48f328035" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:09.454658 16783 raft_consensus.cc:738] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:09.454841 16785 raft_consensus.cc:3054] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.455308 16783 consensus_queue.cc:260] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.456137 16783 raft_consensus.cc:3054] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.456171 17321 raft_consensus.cc:513] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.457921 17321 leader_election.cc:290] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:09.459328 16632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:09.460528 16632 raft_consensus.cc:3054] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.462016 16639 raft_consensus.cc:2463] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:09.463037 16678 leader_election.cc:304] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:09.463217 16785 raft_consensus.cc:2463] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:09.460004 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a21c93f28fc4ab3be409b95204b86dc" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:09.463897 17812 raft_consensus.cc:2798] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:09.464141 16707 raft_consensus.cc:3049] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:09.464386 17812 raft_consensus.cc:695] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:09.464468 16707 raft_consensus.cc:738] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:09.465308 17812 consensus_queue.cc:237] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.465443 16707 consensus_queue.cc:260] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.466207 16534 catalog_manager.cc:5526] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:09.466660 16707 raft_consensus.cc:3054] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.468936 16783 raft_consensus.cc:2463] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:09.473774 16632 raft_consensus.cc:2463] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:09.475129 17288 leader_election.cc:304] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:09.476275 17773 raft_consensus.cc:2798] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:09.476890 17773 raft_consensus.cc:695] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:09.477916 17773 consensus_queue.cc:237] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.485492 16707 raft_consensus.cc:2463] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:09.499923 16537 catalog_manager.cc:5526] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:09.510159 16534 catalog_manager.cc:5526] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:09.716342 17812 raft_consensus.cc:988] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:09.716763 17740 raft_consensus.cc:1076] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:09.718278 16632 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50066
I20250112 20:58:09.718711 16632 raft_consensus.cc:491] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:09.718945 16632 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.729795 17674 raft_consensus.cc:988] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:09.730345 17867 raft_consensus.cc:1076] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:09.726600 16632 raft_consensus.cc:513] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.732062 16639 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "963bd285f345421581941f10dbe8b90c"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50052
I20250112 20:58:09.732295 16632 leader_election.cc:290] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633)
I20250112 20:58:09.732540 16639 raft_consensus.cc:491] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:09.732884 16639 raft_consensus.cc:3054] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.733007 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:09.733570 16707 raft_consensus.cc:3049] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:09.733803 16707 raft_consensus.cc:738] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:09.733783 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
I20250112 20:58:09.734421 17321 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.734575 16707 consensus_queue.cc:260] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.735369 16707 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.738613 16639 raft_consensus.cc:513] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.739327 16707 raft_consensus.cc:2463] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:09.739869 17321 raft_consensus.cc:2463] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:09.740320 16639 leader_election.cc:290] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:09.740459 16602 leader_election.cc:304] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:09.741318 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963bd285f345421581941f10dbe8b90c" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:09.741827 16785 raft_consensus.cc:3049] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:09.742136 16785 raft_consensus.cc:738] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:09.742688 16785 consensus_queue.cc:260] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.741611 17743 raft_consensus.cc:2798] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:09.743490 16785 raft_consensus.cc:3054] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.743925 17743 raft_consensus.cc:695] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:09.744521 17743 consensus_queue.cc:237] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:09.746896 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963bd285f345421581941f10dbe8b90c" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:09.747447 16707 raft_consensus.cc:3054] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:09.748080 16785 raft_consensus.cc:2463] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:09.749233 16605 leader_election.cc:304] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:09.749992 17809 raft_consensus.cc:2798] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:09.750485 17809 raft_consensus.cc:695] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:09.751269 17809 consensus_queue.cc:237] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:09.752494 16707 raft_consensus.cc:2463] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:09.761204 16534 catalog_manager.cc:5526] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:09.776546 16534 catalog_manager.cc:5526] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:09.876950 16785 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:09.878262 17812 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:09.890784 16639 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:09.891942 17812 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:09.927461 16639 raft_consensus.cc:1270] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:09.928375 17812 consensus_queue.cc:1035] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:09.932510 16707 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:09.933517 17773 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:09.939484 16785 raft_consensus.cc:1270] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:09.941963 17879 consensus_queue.cc:1035] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:09.978076 16632 raft_consensus.cc:1270] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:09.979525 17772 consensus_queue.cc:1035] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:10.133875 17321 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:10.134900 17809 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:10.143414 16711 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:10.144564 17743 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:10.160559 16711 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:10.161711 17811 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:10.170751 16785 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:10.172204 17743 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:10.410426 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:10.456097 16785 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:10.456625 16785 raft_consensus.cc:604] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:10.458791 16634 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:10.459328 16634 raft_consensus.cc:604] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:10.461649 16707 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:10.462049 16707 raft_consensus.cc:604] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:10.462986 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250112 20:58:10.464013 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:10.464443 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:10.473042 16707 consensus_queue.cc:237] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:10.478389 16785 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20250112 20:58:10.478616 16678 consensus_peers.cc:487] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1696e724d24e4ebbb185476ec5ff8c09. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:10.479067 16634 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:10.480851 17709 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:10.481563 17879 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:10.487938 17709 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.490842 16785 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.495633 16634 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.504487 16532 catalog_manager.cc:5526] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.516265 16634 consensus_queue.cc:237] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
W20250112 20:58:10.549722 16603 consensus_peers.cc:487] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 807445e11d91471bbfa7a55fcb99f774. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:10.564926 16785 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:10.567167 17743 raft_consensus.cc:988] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:10.568172 17809 raft_consensus.cc:1076] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:10.570170 16707 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:10.572233 17811 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:10.572453 16707 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:59006
I20250112 20:58:10.573056 16707 raft_consensus.cc:491] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:10.573390 16707 raft_consensus.cc:3054] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.573515 17743 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:10.581051 16707 raft_consensus.cc:513] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.583460 16707 leader_election.cc:290] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:10.582768 17743 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.583635 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:10.584260 16783 raft_consensus.cc:3054] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.585346 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c5b79af971640d891e05584ff40dd7b" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:10.585875 16634 raft_consensus.cc:3049] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:10.586158 16634 raft_consensus.cc:738] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:10.586769 16634 consensus_queue.cc:260] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.586640 16785 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.586282 16707 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.587833 16634 raft_consensus.cc:3054] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.597023 16783 raft_consensus.cc:2463] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:10.596951 16711 consensus_queue.cc:237] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:10.598331 16680 leader_election.cc:304] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:10.599334 17879 raft_consensus.cc:2798] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:10.599813 17879 raft_consensus.cc:695] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:10.600647 17879 consensus_queue.cc:237] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.598817 16534 catalog_manager.cc:5526] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:10.602587 16634 raft_consensus.cc:2463] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:10.613734 16783 raft_consensus.cc:1270] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:10.614720 16634 raft_consensus.cc:1270] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:10.615782 17740 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:10.618103 17812 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:10.623811 16678 consensus_peers.cc:487] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c34b0c62d424c9e9f764d74f6baf1ba. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:10.631189 16532 catalog_manager.cc:5526] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:10.631737 17812 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.634789 16783 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.640673 16634 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.644940 16632 consensus_queue.cc:237] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
W20250112 20:58:10.654422 16603 consensus_peers.cc:487] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8a2acedfaa2476596c2ec8bb689dcce. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:10.654886 16783 raft_consensus.cc:1270] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:10.654816 16707 raft_consensus.cc:1270] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:10.656189 17811 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:10.656795 17867 raft_consensus.cc:988] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:10.657208 17867 raft_consensus.cc:1076] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:10.657774 17811 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:10.657297 16534 catalog_manager.cc:5526] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.659756 16632 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50052
I20250112 20:58:10.660154 16632 raft_consensus.cc:491] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:10.660420 16632 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.667445 16632 raft_consensus.cc:513] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.670195 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:10.670817 16785 raft_consensus.cc:3049] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:10.671125 16785 raft_consensus.cc:738] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:10.672715 16632 leader_election.cc:290] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:10.673245 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:10.673748 16711 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.678375 16711 raft_consensus.cc:2463] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:10.679193 16602 leader_election.cc:304] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:10.680744 16785 consensus_queue.cc:260] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.682420 16785 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.679903 17811 raft_consensus.cc:2798] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:10.683395 17811 raft_consensus.cc:695] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:10.689164 17811 consensus_queue.cc:237] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.689419 17899 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.698395 16785 raft_consensus.cc:2463] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:10.697023 16783 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.711073 16711 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.720896 16532 catalog_manager.cc:5526] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:10.727291 16537 catalog_manager.cc:5526] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:10.909909 17740 raft_consensus.cc:988] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d: : Instructing follower c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:10.910416 17879 raft_consensus.cc:1076] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:10.912364 16783 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:39356
I20250112 20:58:10.912868 16783 raft_consensus.cc:491] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:10.913198 16783 raft_consensus.cc:3054] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.919273 16783 raft_consensus.cc:513] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.920665 16783 leader_election.cc:290] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:10.921680 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:10.922258 16634 raft_consensus.cc:3054] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.922572 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d37ea77e5934355b580ed367659c16c" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:10.922744 17911 ts_tablet_manager.cc:927] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:10.923177 16711 raft_consensus.cc:3049] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:10.923467 16711 raft_consensus.cc:738] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:10.924247 16711 consensus_queue.cc:260] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.926613 17911 tablet_copy_client.cc:323] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:10.928325 16711 raft_consensus.cc:3054] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:10.929136 16634 raft_consensus.cc:2463] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:10.929247 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 1696e724d24e4ebbb185476ec5ff8c09 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:10.929814 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 1696e724d24e4ebbb185476ec5ff8c09 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-1696e724d24e4ebbb185476ec5ff8c09
I20250112 20:58:10.930173 16752 leader_election.cc:304] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:10.930961 17674 raft_consensus.cc:2798] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:10.931352 17674 raft_consensus.cc:695] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:10.931995 17674 consensus_queue.cc:237] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:10.938082 16724 tablet_copy_source_session.cc:215] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:10.940479 16711 raft_consensus.cc:2463] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:10.942952 17911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1696e724d24e4ebbb185476ec5ff8c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:10.950681 16532 catalog_manager.cc:5526] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 2 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:10.958971 17911 tablet_copy_client.cc:806] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:10.959569 17911 tablet_copy_client.cc:670] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:10.963016 17911 tablet_copy_client.cc:538] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:10.971503 17911 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:10.991755 17911 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:10.992470 17911 tablet_bootstrap.cc:492] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:10.993058 17911 ts_tablet_manager.cc:1397] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.013s
I20250112 20:58:10.995636 17911 raft_consensus.cc:357] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:10.996452 17911 raft_consensus.cc:738] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:10.997068 17911 consensus_queue.cc:260] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:10.999426 17911 ts_tablet_manager.cc:1428] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20250112 20:58:11.001201 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-1696e724d24e4ebbb185476ec5ff8c09 received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:11.001580 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-1696e724d24e4ebbb185476ec5ff8c09 on tablet 1696e724d24e4ebbb185476ec5ff8c09 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:11.035159 16634 raft_consensus.cc:1270] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:11.036346 17879 consensus_queue.cc:1035] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:11.050972 16783 raft_consensus.cc:1270] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:11.052304 17879 consensus_queue.cc:1035] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:11.103153 17911 ts_tablet_manager.cc:927] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:11.105221 17911 tablet_copy_client.cc:323] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:11.106500 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet d8a2acedfaa2476596c2ec8bb689dcce from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:11.106927 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet d8a2acedfaa2476596c2ec8bb689dcce from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-d8a2acedfaa2476596c2ec8bb689dcce
I20250112 20:58:11.113200 16649 tablet_copy_source_session.cc:215] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:11.119084 17911 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8a2acedfaa2476596c2ec8bb689dcce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:11.127640 17921 ts_tablet_manager.cc:927] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:11.133839 16708 raft_consensus.cc:1270] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:11.135052 17911 tablet_copy_client.cc:806] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:11.135512 17911 tablet_copy_client.cc:670] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:11.135205 17811 consensus_queue.cc:1035] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:11.137284 17921 tablet_copy_client.cc:323] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:11.138787 17911 tablet_copy_client.cc:538] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:11.138753 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 807445e11d91471bbfa7a55fcb99f774 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:11.139461 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 807445e11d91471bbfa7a55fcb99f774 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-807445e11d91471bbfa7a55fcb99f774
I20250112 20:58:11.147125 16649 tablet_copy_source_session.cc:215] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:11.151876 17921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 807445e11d91471bbfa7a55fcb99f774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:11.153267 16783 raft_consensus.cc:1270] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:11.147855 17911 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:11.166427 17811 consensus_queue.cc:1035] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:11.192178 17926 ts_tablet_manager.cc:927] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:11.194671 17921 tablet_copy_client.cc:806] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:11.195379 17921 tablet_copy_client.cc:670] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:11.196030 17926 tablet_copy_client.cc:323] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:11.198719 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 6c34b0c62d424c9e9f764d74f6baf1ba from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:11.199203 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 6c34b0c62d424c9e9f764d74f6baf1ba from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-6c34b0c62d424c9e9f764d74f6baf1ba
I20250112 20:58:11.203787 17921 tablet_copy_client.cc:538] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:11.206809 16724 tablet_copy_source_session.cc:215] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:11.209493 17926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c34b0c62d424c9e9f764d74f6baf1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:11.215900 17921 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:11.229050 17926 tablet_copy_client.cc:806] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:11.229605 17926 tablet_copy_client.cc:670] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:11.233264 17926 tablet_copy_client.cc:538] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:11.234596 17911 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:11.235469 17911 tablet_bootstrap.cc:492] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:11.236029 17911 ts_tablet_manager.cc:1397] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.088s user 0.016s sys 0.011s
I20250112 20:58:11.238385 17911 raft_consensus.cc:357] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:11.239180 17911 raft_consensus.cc:738] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:11.239784 17911 consensus_queue.cc:260] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:11.243707 17911 ts_tablet_manager.cc:1428] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20250112 20:58:11.245164 17926 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:11.245576 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-d8a2acedfaa2476596c2ec8bb689dcce received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:11.245987 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-d8a2acedfaa2476596c2ec8bb689dcce on tablet d8a2acedfaa2476596c2ec8bb689dcce with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:11.253329 17921 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:11.254093 17921 tablet_bootstrap.cc:492] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:11.254673 17921 ts_tablet_manager.cc:1397] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.039s user 0.013s sys 0.008s
I20250112 20:58:11.256840 17921 raft_consensus.cc:357] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:11.257699 17921 raft_consensus.cc:738] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:11.258260 17921 consensus_queue.cc:260] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:11.263124 17921 ts_tablet_manager.cc:1428] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20250112 20:58:11.264472 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-807445e11d91471bbfa7a55fcb99f774 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:11.264940 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-807445e11d91471bbfa7a55fcb99f774 on tablet 807445e11d91471bbfa7a55fcb99f774 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:11.278999 17926 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:11.279930 17926 tablet_bootstrap.cc:492] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:11.280612 17926 ts_tablet_manager.cc:1397] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.039s user 0.018s sys 0.001s
I20250112 20:58:11.283591 17926 raft_consensus.cc:357] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:11.284382 17926 raft_consensus.cc:738] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:11.284979 17926 consensus_queue.cc:260] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:11.288913 17926 ts_tablet_manager.cc:1428] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250112 20:58:11.290505 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-6c34b0c62d424c9e9f764d74f6baf1ba received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:11.290877 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-6c34b0c62d424c9e9f764d74f6baf1ba on tablet 6c34b0c62d424c9e9f764d74f6baf1ba with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:11.407131 16711 raft_consensus.cc:1270] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:11.409161 16639 raft_consensus.cc:1270] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:11.410002 17924 consensus_queue.cc:1035] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20250112 20:58:11.412173 17867 consensus_queue.cc:1035] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:11.440691 17321 raft_consensus.cc:1212] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250112 20:58:11.509085 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:11.556430 16783 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:11.557013 16783 raft_consensus.cc:604] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:11.559146 16707 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7a13e4117fd64b15a4cd60943e638999"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:11.559724 16707 raft_consensus.cc:604] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:11.561887 16639 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:11.562386 16639 raft_consensus.cc:604] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:11.564466 16639 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e23f9778f8494e11b9d942035720e363"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:11.564996 16639 raft_consensus.cc:604] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:11.565901 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20250112 20:58:11.567288 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:11.567781 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:11.667994 17321 raft_consensus.cc:1212] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:11.682703 17321 raft_consensus.cc:1212] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:11.719734 17812 raft_consensus.cc:988] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:11.720214 17879 raft_consensus.cc:1076] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:11.720805 17324 raft_consensus.cc:1212] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:11.721719 17321 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7a13e4117fd64b15a4cd60943e638999"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53256
I20250112 20:58:11.722244 17321 raft_consensus.cc:491] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:11.722582 17321 raft_consensus.cc:3054] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.728790 17321 raft_consensus.cc:513] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.731092 16708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a13e4117fd64b15a4cd60943e638999" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:11.731199 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a13e4117fd64b15a4cd60943e638999" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:11.731642 16708 raft_consensus.cc:3049] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:11.731778 16785 raft_consensus.cc:3054] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.731920 16708 raft_consensus.cc:738] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:11.732581 16708 consensus_queue.cc:260] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.733606 16708 raft_consensus.cc:3054] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.739548 17321 leader_election.cc:290] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:11.739948 16785 raft_consensus.cc:2463] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:11.741016 17290 leader_election.cc:304] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:11.741658 17895 raft_consensus.cc:2798] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:11.741806 16708 raft_consensus.cc:2463] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:11.742080 17895 raft_consensus.cc:695] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:11.743052 17895 consensus_queue.cc:237] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.750672 16537 catalog_manager.cc:5526] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:11.752825 17910 raft_consensus.cc:988] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:11.753270 17743 raft_consensus.cc:1076] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:11.754963 16785 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:39342
I20250112 20:58:11.755439 16785 raft_consensus.cc:491] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:11.755741 16785 raft_consensus.cc:3054] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.761926 16785 raft_consensus.cc:513] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.763752 16785 leader_election.cc:290] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633)
I20250112 20:58:11.766229 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:11.766395 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82159eb8aadb4973b65d7edc4f417d84" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
I20250112 20:58:11.766759 16639 raft_consensus.cc:3049] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:11.766868 17321 raft_consensus.cc:3054] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.767083 16639 raft_consensus.cc:738] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:11.767792 16639 consensus_queue.cc:260] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.769004 16639 raft_consensus.cc:3054] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.773241 17321 raft_consensus.cc:2463] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:11.774297 16752 leader_election.cc:304] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:11.775053 17674 raft_consensus.cc:2798] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:11.775475 17674 raft_consensus.cc:695] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:11.776147 17674 consensus_queue.cc:237] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.779677 16639 raft_consensus.cc:2463] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:11.791158 16537 catalog_manager.cc:5526] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 2 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:11.930096 17899 raft_consensus.cc:988] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:11.930572 17899 raft_consensus.cc:1076] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:11.932224 16711 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e23f9778f8494e11b9d942035720e363"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:59006
I20250112 20:58:11.932742 16711 raft_consensus.cc:491] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:11.933099 16711 raft_consensus.cc:3054] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.938060 16711 raft_consensus.cc:513] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.939468 16711 leader_election.cc:290] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633)
I20250112 20:58:11.940274 17321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23f9778f8494e11b9d942035720e363" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
I20250112 20:58:11.940762 17321 raft_consensus.cc:3054] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.941187 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e23f9778f8494e11b9d942035720e363" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:11.941809 16639 raft_consensus.cc:3049] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:11.942130 16639 raft_consensus.cc:738] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:11.944387 16639 consensus_queue.cc:260] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.945278 16639 raft_consensus.cc:3054] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.945744 17321 raft_consensus.cc:2463] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:11.946529 16678 leader_election.cc:304] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters:
I20250112 20:58:11.947192 17812 raft_consensus.cc:2798] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:11.947551 17812 raft_consensus.cc:695] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:11.948314 17812 consensus_queue.cc:237] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.949748 16639 raft_consensus.cc:2463] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:11.955271 17674 raft_consensus.cc:988] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:11.955687 17924 raft_consensus.cc:1076] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:11.957279 16639 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50052
I20250112 20:58:11.957787 16639 raft_consensus.cc:491] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:11.958104 16639 raft_consensus.cc:3054] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.959947 16537 catalog_manager.cc:5526] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:11.963966 16639 raft_consensus.cc:513] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:11.968302 16639 leader_election.cc:290] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:11.969452 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:11.969972 16783 raft_consensus.cc:3049] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:11.970204 16783 raft_consensus.cc:738] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:11.970742 16783 consensus_queue.cc:260] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:11.971629 17879 raft_consensus.cc:1059] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:11.971889 16783 raft_consensus.cc:3054] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.973495 17879 consensus_queue.cc:237] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:11.979475 16639 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:11.980050 17321 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:11.980417 16787 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:11.980912 17812 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:11.981917 17740 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:11.984737 17812 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:11.984122 16783 raft_consensus.cc:2463] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:11.989748 16605 leader_election.cc:304] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:11.990654 17743 raft_consensus.cc:2798] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:11.991076 17743 raft_consensus.cc:695] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:11.987529 16708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7ad2e8f5064918bd15be2a608c1db4" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:11.992094 16708 raft_consensus.cc:3054] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:11.991883 17743 consensus_queue.cc:237] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:12.003916 16708 raft_consensus.cc:2463] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:12.006306 17740 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.008191 17321 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.015031 16783 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.034643 16534 catalog_manager.cc:5526] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:12.031877 16537 catalog_manager.cc:5526] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:12.031311 16639 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.064968 16707 consensus_queue.cc:237] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.070907 16783 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:12.070907 17324 raft_consensus.cc:1270] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:12.072117 17941 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:12.073774 17740 consensus_queue.cc:1035] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:12.078219 17941 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.079595 16785 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.087054 17321 raft_consensus.cc:2949] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.094641 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 1696e724d24e4ebbb185476ec5ff8c09 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:12.099274 16537 catalog_manager.cc:5526] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 4 to 5, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.110885 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 1696e724d24e4ebbb185476ec5ff8c09 with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:12.113221 17812 raft_consensus.cc:1059] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:12.113057 17946 tablet_replica.cc:331] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:12.114218 17946 raft_consensus.cc:2238] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:12.114796 17946 raft_consensus.cc:2267] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:12.115366 17812 consensus_queue.cc:237] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.117954 17946 ts_tablet_manager.cc:1905] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:12.138674 17321 raft_consensus.cc:1270] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.144098 16785 raft_consensus.cc:1270] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.144740 17812 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:58:12.161270 17941 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:12.146675 16634 raft_consensus.cc:1270] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.165506 17941 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:12.168692 17946 ts_tablet_manager.cc:1918] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250112 20:58:12.169572 17946 log.cc:1198] T 1696e724d24e4ebbb185476ec5ff8c09 P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/1696e724d24e4ebbb185476ec5ff8c09
I20250112 20:58:12.173316 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 1696e724d24e4ebbb185476ec5ff8c09 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:12.193732 16632 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:12.195222 17924 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:12.203019 16783 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.204164 16634 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.205716 17812 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.218544 16533 catalog_manager.cc:5526] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.221048 17321 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.244362 17322 raft_consensus.cc:1270] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:12.245528 17674 consensus_queue.cc:1035] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:12.241863 16708 consensus_queue.cc:237] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.261559 16783 raft_consensus.cc:1270] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:12.263082 17321 raft_consensus.cc:1270] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:12.264998 17941 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:12.266609 17941 consensus_queue.cc:1035] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:12.272482 17879 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.281433 16785 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.283479 16783 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:12.285215 17896 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:12.287556 17899 raft_consensus.cc:1059] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:12.296710 17899 consensus_queue.cc:237] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.307093 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 6c34b0c62d424c9e9f764d74f6baf1ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:12.311678 17743 raft_consensus.cc:1059] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:12.317193 17321 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.318704 16707 raft_consensus.cc:1270] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:12.320363 17896 consensus_queue.cc:1035] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:12.320886 16707 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.313997 17743 consensus_queue.cc:237] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.328217 16537 catalog_manager.cc:5526] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:12.330135 17910 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250112 20:58:12.330789 16678 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:41463, user_credentials={real_user=slave}} blocked reactor thread for 55997.8us
I20250112 20:58:12.332553 17322 raft_consensus.cc:2949] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.337533 16785 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.339630 17899 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:58:12.352197 17899 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:12.374125 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c34b0c62d424c9e9f764d74f6baf1ba with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:12.375090 17946 tablet_replica.cc:331] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:12.376009 17946 raft_consensus.cc:2238] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:12.376809 17946 raft_consensus.cc:2267] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:12.379777 17946 ts_tablet_manager.cc:1905] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:12.409178 16787 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:12.413944 17968 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:12.424897 17946 ts_tablet_manager.cc:1918] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:12.425562 17946 log.cc:1198] T 6c34b0c62d424c9e9f764d74f6baf1ba P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/6c34b0c62d424c9e9f764d74f6baf1ba
I20250112 20:58:12.428161 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 6c34b0c62d424c9e9f764d74f6baf1ba (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
W20250112 20:58:12.432808 16602 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:38087, user_credentials={real_user=slave}} blocked reactor thread for 110445us
I20250112 20:58:12.448550 16708 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:12.450579 17968 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:12.454653 17322 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:12.456068 17899 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.457332 17812 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:12.459268 17322 raft_consensus.cc:1270] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.460377 17743 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:12.462664 16709 raft_consensus.cc:1270] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.463176 16632 raft_consensus.cc:1270] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:12.464530 17740 consensus_queue.cc:1035] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:12.472801 16708 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.464025 17743 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20250112 20:58:12.500932 16785 raft_consensus.cc:1270] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:12.502650 17899 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20250112 20:58:12.557696 17972 log.cc:926] Time spent T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d: Append to log took a long time: real 0.057s user 0.000s sys 0.003s
I20250112 20:58:12.557621 17743 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.556785 16534 catalog_manager.cc:5526] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:12.572705 17324 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.562314 17321 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.563570 16783 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.590126 16707 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.559351 16787 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.608731 16533 catalog_manager.cc:5526] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.618312 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:12.639699 16634 consensus_queue.cc:237] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.646651 16785 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:12.647562 17321 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:12.647656 16636 consensus_queue.cc:237] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.648929 17971 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:12.651705 17910 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:12.653710 17322 raft_consensus.cc:1270] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:12.654699 16711 raft_consensus.cc:1270] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:12.654801 17960 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:12.655887 17971 consensus_queue.cc:1035] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:12.657281 17960 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.662192 17322 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.662572 17743 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.671695 17324 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.675412 16707 raft_consensus.cc:2949] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.686496 16785 raft_consensus.cc:2949] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.700232 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet d8a2acedfaa2476596c2ec8bb689dcce with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:12.701673 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 807445e11d91471bbfa7a55fcb99f774 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:12.702205 16708 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:12.702775 16708 raft_consensus.cc:604] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:12.707337 16537 catalog_manager.cc:5526] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.708549 16783 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:12.709174 16783 raft_consensus.cc:604] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:12.713352 16636 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:12.717686 16636 raft_consensus.cc:604] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:12.715251 16534 catalog_manager.cc:5526] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:12.719100 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250112 20:58:12.720467 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:12.721046 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:12.727780 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 807445e11d91471bbfa7a55fcb99f774 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:12.729797 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet d8a2acedfaa2476596c2ec8bb689dcce with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:12.736891 17978 tablet_replica.cc:331] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:12.738456 17978 raft_consensus.cc:2238] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:12.739845 17978 raft_consensus.cc:2267] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:12.742918 17978 ts_tablet_manager.cc:1905] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:12.744386 17979 tablet_replica.cc:331] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:12.745509 17979 raft_consensus.cc:2238] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:12.746220 17979 raft_consensus.cc:2267] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:12.752173 17979 ts_tablet_manager.cc:1905] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:12.753408 17910 raft_consensus.cc:988] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:12.753844 17899 raft_consensus.cc:1076] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:12.755796 16783 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:39342
I20250112 20:58:12.756301 16783 raft_consensus.cc:491] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:12.756635 16783 raft_consensus.cc:3054] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:12.760047 17978 ts_tablet_manager.cc:1918] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:12.760463 17978 log.cc:1198] T 807445e11d91471bbfa7a55fcb99f774 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/807445e11d91471bbfa7a55fcb99f774
I20250112 20:58:12.761741 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 807445e11d91471bbfa7a55fcb99f774 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:12.762313 16783 raft_consensus.cc:513] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.764037 16783 leader_election.cc:290] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633)
I20250112 20:58:12.765002 17324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
I20250112 20:58:12.765123 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef14661bd7346f68101fe2a0fb064bb" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:12.765575 17324 raft_consensus.cc:3054] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:12.765611 16636 raft_consensus.cc:3049] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:12.766112 16636 raft_consensus.cc:738] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:12.766790 16636 consensus_queue.cc:260] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.767896 16636 raft_consensus.cc:3054] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:12.771394 17324 raft_consensus.cc:2463] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:12.772351 16752 leader_election.cc:304] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:12.773074 17924 raft_consensus.cc:2798] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:12.773540 17924 raft_consensus.cc:695] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:12.773599 16636 raft_consensus.cc:2463] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:12.774416 17924 consensus_queue.cc:237] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:12.777652 17979 ts_tablet_manager.cc:1918] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:12.778548 17979 log.cc:1198] T d8a2acedfaa2476596c2ec8bb689dcce P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/d8a2acedfaa2476596c2ec8bb689dcce
I20250112 20:58:12.783192 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet d8a2acedfaa2476596c2ec8bb689dcce (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:12.793257 16534 catalog_manager.cc:5526] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 2 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:13.035553 17956 raft_consensus.cc:988] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:13.036293 17956 raft_consensus.cc:1076] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:13.037582 17324 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53256
I20250112 20:58:13.038038 17324 raft_consensus.cc:491] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:13.038359 17324 raft_consensus.cc:3054] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:13.043033 17324 raft_consensus.cc:513] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:13.044821 17324 leader_election.cc:290] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:13.045001 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:13.045535 16707 raft_consensus.cc:3049] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:13.045593 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "142fbcc35085432abdb80a4fad031c67" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:13.045867 16707 raft_consensus.cc:738] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:13.046118 16634 raft_consensus.cc:3054] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:13.046510 16707 consensus_queue.cc:260] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:13.047461 16707 raft_consensus.cc:3054] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:13.052474 16634 raft_consensus.cc:2463] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:13.053278 16707 raft_consensus.cc:2463] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 2.
I20250112 20:58:13.053359 17288 leader_election.cc:304] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:13.054248 17895 raft_consensus.cc:2798] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:13.054702 17895 raft_consensus.cc:695] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:13.055426 17895 consensus_queue.cc:237] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:13.063455 16534 catalog_manager.cc:5526] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 2 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:13.136987 17674 raft_consensus.cc:988] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:13.137347 17981 raft_consensus.cc:1076] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:13.138414 16634 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50052
I20250112 20:58:13.138844 16634 raft_consensus.cc:491] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:13.139071 16634 raft_consensus.cc:3054] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:13.143234 16634 raft_consensus.cc:513] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:13.144500 16634 leader_election.cc:290] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:13.145072 16707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:13.145392 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1cfa7b34e542098452925ed942c2bb" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:13.145771 16707 raft_consensus.cc:3054] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:13.145865 16785 raft_consensus.cc:3049] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:13.146181 16785 raft_consensus.cc:738] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:13.146821 16785 consensus_queue.cc:260] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:13.147854 16785 raft_consensus.cc:3054] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:13.153414 16707 raft_consensus.cc:2463] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:13.154364 16602 leader_election.cc:304] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:13.155094 17910 raft_consensus.cc:2798] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:13.155598 17910 raft_consensus.cc:695] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:13.159865 16785 raft_consensus.cc:2463] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:13.159022 17910 consensus_queue.cc:237] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:13.172122 16534 catalog_manager.cc:5526] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:13.244738 16634 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:13.246063 17674 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:13.257053 17322 raft_consensus.cc:1270] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:13.258471 17674 consensus_queue.cc:1035] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:13.548274 16708 raft_consensus.cc:1270] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:13.550052 17910 consensus_queue.cc:1035] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:13.567159 16787 raft_consensus.cc:1270] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:13.568806 17971 consensus_queue.cc:1035] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:13.665521 16708 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:13.666626 17895 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:13.679590 16636 raft_consensus.cc:1270] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:13.681012 17996 consensus_queue.cc:1035] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250112 20:58:13.795218 16787 consensus_queue.cc:237] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:13.802054 16707 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250112 20:58:13.802124 16752 consensus_peers.cc:487] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e161101273b4b61a7235af454d77ca1. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:13.803606 17999 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:13.804350 16636 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:13.806077 17999 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:13.811897 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:13.813628 17924 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.818256 16707 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.823194 16634 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.827301 16636 consensus_queue.cc:237] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:13.837513 16534 catalog_manager.cc:5526] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.844318 16707 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:13.845595 17971 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20250112 20:58:13.848320 16603 consensus_peers.cc:487] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f7ad2e8f5064918bd15be2a608c1db4. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:13.853567 16783 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:13.860126 17971 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:13.863842 17971 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.869941 16707 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.912169 16783 consensus_queue.cc:237] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:13.914269 16787 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:58:13.925849 16752 consensus_peers.cc:487] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1d0b3432b644821ba556f59a893d40a. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:13.926726 16707 raft_consensus.cc:1270] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:13.927214 16532 catalog_manager.cc:5526] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.927963 17674 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:13.928833 16636 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:13.929544 16636 raft_consensus.cc:604] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:13.940039 16785 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:13.940585 16785 raft_consensus.cc:604] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:13.946153 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:58:13.948112 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:13.949257 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:13.954496 17910 raft_consensus.cc:988] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:13.954990 17910 raft_consensus.cc:1076] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:13.956573 16708 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:59006
I20250112 20:58:13.957103 16708 raft_consensus.cc:491] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:13.957448 16708 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:13.961268 16634 raft_consensus.cc:1270] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:13.962508 17924 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:13.968534 18004 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.977526 16636 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.982005 16707 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:13.991338 16711 consensus_queue.cc:237] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:13.987771 16708 raft_consensus.cc:513] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:14.029884 16634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:14.030441 16634 raft_consensus.cc:3049] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:58:14.030649 16634 raft_consensus.cc:738] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:14.031293 16634 consensus_queue.cc:260] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:14.032326 16634 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Advancing to term 3
W20250112 20:58:14.032899 16678 consensus_peers.cc:487] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bdb0f016b8946c582e26ab6e02cb9b1. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:14.038864 16787 raft_consensus.cc:1270] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:14.040375 17995 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:14.039871 16534 catalog_manager.cc:5526] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:14.044139 16634 raft_consensus.cc:2463] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 3.
I20250112 20:58:14.045936 16678 leader_election.cc:304] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:14.046130 16634 raft_consensus.cc:1270] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:14.053747 17995 raft_consensus.cc:2798] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:14.054236 17995 raft_consensus.cc:695] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 3 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:14.054951 17995 consensus_queue.cc:237] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:14.055049 17993 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:14.061998 16708 leader_election.cc:290] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 3 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:14.062093 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10e3fd35bff34f4bbaf6763ca8651656" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:14.062907 16785 raft_consensus.cc:3054] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:14.066251 17941 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:14.072942 16787 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:14.086427 16785 raft_consensus.cc:2463] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 3.
I20250112 20:58:14.098587 16636 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:14.106276 16534 catalog_manager.cc:5526] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 2 to 3, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 3 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:14.107968 16534 catalog_manager.cc:5526] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:14.296401 18017 ts_tablet_manager.cc:927] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:14.297809 17924 raft_consensus.cc:988] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:14.298413 17924 raft_consensus.cc:1076] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:14.298447 18017 tablet_copy_client.cc:323] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:58:14.299943 16634 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50052
I20250112 20:58:14.300457 16634 raft_consensus.cc:491] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:14.300789 16634 raft_consensus.cc:3054] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:14.306236 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet 8e161101273b4b61a7235af454d77ca1 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:58:14.306824 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet 8e161101273b4b61a7235af454d77ca1 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-8e161101273b4b61a7235af454d77ca1
I20250112 20:58:14.318035 18019 ts_tablet_manager.cc:927] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:14.318416 16634 raft_consensus.cc:513] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.322126 18019 tablet_copy_client.cc:323] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:14.323173 16634 leader_election.cc:290] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:14.324313 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 3f7ad2e8f5064918bd15be2a608c1db4 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:14.325554 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 3f7ad2e8f5064918bd15be2a608c1db4 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-3f7ad2e8f5064918bd15be2a608c1db4
I20250112 20:58:14.324652 16708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:14.326087 16708 raft_consensus.cc:3054] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:14.324920 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e161101273b4b61a7235af454d77ca1" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:14.333250 16783 raft_consensus.cc:3049] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:14.333542 16783 raft_consensus.cc:738] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:14.334188 16783 consensus_queue.cc:260] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.335180 16783 raft_consensus.cc:3054] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:14.336870 16649 tablet_copy_source_session.cc:215] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:14.336925 16708 raft_consensus.cc:2463] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:14.321576 16798 tablet_copy_source_session.cc:215] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:14.338232 16602 leader_election.cc:304] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:14.338804 17910 raft_consensus.cc:2798] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:14.339258 17910 raft_consensus.cc:695] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:14.340122 17910 consensus_queue.cc:237] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.341887 16783 raft_consensus.cc:2463] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:14.343375 18017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e161101273b4b61a7235af454d77ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:14.349874 18019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f7ad2e8f5064918bd15be2a608c1db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:14.365737 18019 tablet_copy_client.cc:806] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:14.366485 18019 tablet_copy_client.cc:670] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:14.372380 18019 tablet_copy_client.cc:538] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:14.365469 16537 catalog_manager.cc:5526] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:14.376528 18017 tablet_copy_client.cc:806] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:14.377158 18017 tablet_copy_client.cc:670] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:14.380755 18017 tablet_copy_client.cc:538] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:14.381942 18019 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:14.392840 18017 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:14.430130 18017 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:14.431063 18017 tablet_bootstrap.cc:492] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:14.431757 18017 ts_tablet_manager.cc:1397] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.039s user 0.014s sys 0.013s
I20250112 20:58:14.431958 18019 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:14.432896 18019 tablet_bootstrap.cc:492] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:14.433591 18019 ts_tablet_manager.cc:1397] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.052s user 0.028s sys 0.004s
I20250112 20:58:14.434898 18017 raft_consensus.cc:357] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.436031 18017 raft_consensus.cc:738] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:14.436424 18019 raft_consensus.cc:357] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.436744 18017 consensus_queue.cc:260] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.437907 18019 raft_consensus.cc:738] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:14.438639 18019 consensus_queue.cc:260] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.440367 18017 ts_tablet_manager.cc:1428] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20250112 20:58:14.441545 18019 ts_tablet_manager.cc:1428] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.008s user 0.002s sys 0.005s
I20250112 20:58:14.441519 16634 raft_consensus.cc:1270] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:14.443480 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-3f7ad2e8f5064918bd15be2a608c1db4 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:14.443328 17993 consensus_queue.cc:1035] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:14.443918 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-3f7ad2e8f5064918bd15be2a608c1db4 on tablet 3f7ad2e8f5064918bd15be2a608c1db4 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:14.447237 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-8e161101273b4b61a7235af454d77ca1 received from {username='slave'} at 127.0.0.1:51714
I20250112 20:58:14.447678 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-8e161101273b4b61a7235af454d77ca1 on tablet 8e161101273b4b61a7235af454d77ca1 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:14.459173 18017 ts_tablet_manager.cc:927] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:14.461444 16783 raft_consensus.cc:1270] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:14.465382 17941 consensus_queue.cc:1035] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:14.474108 18017 tablet_copy_client.cc:323] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.131:38713
I20250112 20:58:14.475878 16798 tablet_copy_service.cc:140] P c29b243acf2b4f248606563fc96efc7d: Received BeginTabletCopySession request for tablet b1d0b3432b644821ba556f59a893d40a from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:51714)
I20250112 20:58:14.476397 16798 tablet_copy_service.cc:161] P c29b243acf2b4f248606563fc96efc7d: Beginning new tablet copy session on tablet b1d0b3432b644821ba556f59a893d40a from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:51714: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-b1d0b3432b644821ba556f59a893d40a
I20250112 20:58:14.494205 16798 tablet_copy_source_session.cc:215] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:14.498142 18017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d0b3432b644821ba556f59a893d40a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:14.519088 18017 tablet_copy_client.cc:806] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:14.519785 18017 tablet_copy_client.cc:670] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:14.522650 18017 tablet_copy_client.cc:538] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:14.530079 18017 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:14.552428 18017 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:14.553297 18017 tablet_bootstrap.cc:492] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:14.553812 18017 ts_tablet_manager.cc:1397] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.024s user 0.018s sys 0.005s
I20250112 20:58:14.557832 18017 raft_consensus.cc:357] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.558516 18017 raft_consensus.cc:738] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:14.559033 18017 consensus_queue.cc:260] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.561477 18017 ts_tablet_manager.cc:1428] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250112 20:58:14.563350 16798 tablet_copy_service.cc:342] P c29b243acf2b4f248606563fc96efc7d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-b1d0b3432b644821ba556f59a893d40a received from {username='slave'} at 127.0.0.1:51714
I20250112 20:58:14.563769 16798 tablet_copy_service.cc:434] P c29b243acf2b4f248606563fc96efc7d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-b1d0b3432b644821ba556f59a893d40a on tablet b1d0b3432b644821ba556f59a893d40a with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:14.611971 18017 ts_tablet_manager.cc:927] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:14.613534 18017 tablet_copy_client.cc:323] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:14.614918 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 2bdb0f016b8946c582e26ab6e02cb9b1 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:14.615355 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 2bdb0f016b8946c582e26ab6e02cb9b1 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-2bdb0f016b8946c582e26ab6e02cb9b1
I20250112 20:58:14.621595 16724 tablet_copy_source_session.cc:215] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:14.623893 18017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bdb0f016b8946c582e26ab6e02cb9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:14.636842 18017 tablet_copy_client.cc:806] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:14.637382 18017 tablet_copy_client.cc:670] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:14.640267 18017 tablet_copy_client.cc:538] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:14.649955 18017 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:14.664810 18017 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:14.665438 18017 tablet_bootstrap.cc:492] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:14.665942 18017 ts_tablet_manager.cc:1397] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.004s
I20250112 20:58:14.668221 18017 raft_consensus.cc:357] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.668963 18017 raft_consensus.cc:738] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:14.669508 18017 consensus_queue.cc:260] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:14.674607 18017 ts_tablet_manager.cc:1428] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250112 20:58:14.676019 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-2bdb0f016b8946c582e26ab6e02cb9b1 received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:14.676429 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-2bdb0f016b8946c582e26ab6e02cb9b1 on tablet 2bdb0f016b8946c582e26ab6e02cb9b1 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:14.757354 17321 raft_consensus.cc:1212] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250112 20:58:14.858033 17321 raft_consensus.cc:1212] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:14.860318 17321 raft_consensus.cc:3054] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Advancing to term 2
I20250112 20:58:14.865434 16783 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:14.866883 17910 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:14.871044 17321 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:14.872303 17910 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:14.898324 16708 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:14.899627 17971 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:14.909092 17968 raft_consensus.cc:1059] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:14.911399 17968 consensus_queue.cc:237] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:14.918300 16787 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:14.918438 16711 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:14.919903 17971 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20250112 20:58:14.928642 17968 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:58:14.930917 17322 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:14.938928 17971 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:14.955027 17971 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:14.956020 16711 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:14.972239 17324 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:14.979594 16787 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:14.988498 16532 catalog_manager.cc:5526] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:15.028673 16632 consensus_queue.cc:237] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.033691 17324 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:15.034090 16787 raft_consensus.cc:1270] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:15.034893 17910 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:15.035753 17971 consensus_queue.cc:1035] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:15.041715 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:15.041766 17910 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.042846 17324 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.050894 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 8e161101273b4b61a7235af454d77ca1 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:15.055083 16787 raft_consensus.cc:2949] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.059887 16534 catalog_manager.cc:5526] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 4 to 5, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:15.084409 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e161101273b4b61a7235af454d77ca1 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:15.094311 18041 tablet_replica.cc:331] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:15.095402 18041 raft_consensus.cc:2238] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:15.096101 18041 raft_consensus.cc:2267] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:15.099620 18041 ts_tablet_manager.cc:1905] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:15.114616 18041 ts_tablet_manager.cc:1918] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250112 20:58:15.115010 18041 log.cc:1198] T 8e161101273b4b61a7235af454d77ca1 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/8e161101273b4b61a7235af454d77ca1
I20250112 20:58:15.117219 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 8e161101273b4b61a7235af454d77ca1 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:15.118726 16632 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:15.119287 16632 raft_consensus.cc:604] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:15.121592 16708 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "81567654a26f435795e38a761d26bfa7"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:15.122013 16708 raft_consensus.cc:604] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:15.124819 16708 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:15.125319 16708 raft_consensus.cc:604] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:15.126302 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250112 20:58:15.127679 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:15.128223 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:15.198096 17324 raft_consensus.cc:1212] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:15.292742 17910 raft_consensus.cc:988] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:15.293131 17971 raft_consensus.cc:1076] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:15.294323 16783 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:39342
I20250112 20:58:15.294754 16783 raft_consensus.cc:491] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:15.295071 16783 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.303674 16783 raft_consensus.cc:513] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.305425 16783 leader_election.cc:290] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633)
I20250112 20:58:15.306686 17324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
I20250112 20:58:15.306733 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:15.307320 17324 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.307412 16636 raft_consensus.cc:3049] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:15.307693 16636 raft_consensus.cc:738] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:15.308277 16636 consensus_queue.cc:260] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.309429 16636 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.312685 17324 raft_consensus.cc:2463] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:15.313768 16752 leader_election.cc:304] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:15.314364 18032 raft_consensus.cc:2798] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:15.314744 18032 raft_consensus.cc:695] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:15.315838 18032 consensus_queue.cc:237] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.314708 16636 raft_consensus.cc:2463] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:15.326728 16537 catalog_manager.cc:5526] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 2 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:15.353615 17941 raft_consensus.cc:988] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d: : Instructing follower c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:15.354110 17941 raft_consensus.cc:1076] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:15.356459 16783 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:39356
I20250112 20:58:15.357292 18036 raft_consensus.cc:1059] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:15.357499 16783 raft_consensus.cc:491] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:15.357916 16783 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.359505 18036 consensus_queue.cc:237] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.363523 16783 raft_consensus.cc:513] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:15.365772 17322 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:15.366202 16787 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:15.367278 16711 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:15.367417 17910 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:15.369251 17971 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:15.370512 17968 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:15.375476 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:15.376061 16636 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.384625 17968 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.386162 17321 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.390980 16636 raft_consensus.cc:2463] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:15.390336 16708 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.400985 16783 leader_election.cc:290] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:15.422264 16783 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.423115 16709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:15.423714 16709 raft_consensus.cc:3049] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:15.423977 16709 raft_consensus.cc:738] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:15.424651 16709 consensus_queue.cc:260] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:15.425662 16709 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.431823 16709 raft_consensus.cc:2463] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:15.437112 16537 catalog_manager.cc:5526] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:15.448729 16751 leader_election.cc:304] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:15.449452 17924 raft_consensus.cc:2798] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:15.449875 17924 raft_consensus.cc:695] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:15.450600 17924 consensus_queue.cc:237] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:15.451833 16636 consensus_queue.cc:237] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.459755 17321 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:15.459755 16708 raft_consensus.cc:1270] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:15.461489 17968 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:15.466804 17968 consensus_queue.cc:1035] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:15.473778 17971 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.481355 16534 catalog_manager.cc:5526] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 2 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:15.488189 17322 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.493422 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 3f7ad2e8f5064918bd15be2a608c1db4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:15.498445 18004 raft_consensus.cc:1059] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:15.498566 16533 catalog_manager.cc:5526] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:15.500957 18004 consensus_queue.cc:237] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.517880 17324 raft_consensus.cc:1270] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:15.518368 16711 raft_consensus.cc:1270] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:15.521860 16634 raft_consensus.cc:1270] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:15.522784 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7ad2e8f5064918bd15be2a608c1db4 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:15.523662 17924 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:15.525398 17924 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:15.534885 17924 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:15.558480 18052 tablet_replica.cc:331] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:15.559657 18052 raft_consensus.cc:2238] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:15.560387 18052 raft_consensus.cc:2267] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:15.563037 17995 raft_consensus.cc:988] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:15.563561 17995 raft_consensus.cc:1076] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:15.565007 16636 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "81567654a26f435795e38a761d26bfa7"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50066
I20250112 20:58:15.565496 16636 raft_consensus.cc:491] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:15.565804 16636 raft_consensus.cc:3054] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.567173 18052 ts_tablet_manager.cc:1905] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:15.569345 16708 raft_consensus.cc:2949] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.574344 16636 raft_consensus.cc:513] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:15.586881 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:15.587477 16787 raft_consensus.cc:3054] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.585860 16636 leader_election.cc:290] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:15.593852 16787 raft_consensus.cc:2463] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:15.595029 16605 leader_election.cc:304] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:15.595727 17971 raft_consensus.cc:2798] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:15.596158 17971 raft_consensus.cc:695] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:15.596918 17971 consensus_queue.cc:237] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:15.596257 18004 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.601032 18052 ts_tablet_manager.cc:1918] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250112 20:58:15.601523 18052 log.cc:1198] T 3f7ad2e8f5064918bd15be2a608c1db4 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/3f7ad2e8f5064918bd15be2a608c1db4
I20250112 20:58:15.603297 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet 3f7ad2e8f5064918bd15be2a608c1db4 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:15.604152 17322 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.607497 16634 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.622757 16709 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.635332 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81567654a26f435795e38a761d26bfa7" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:15.635816 16711 raft_consensus.cc:3049] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:15.636030 16711 raft_consensus.cc:738] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:15.636564 16711 consensus_queue.cc:260] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:15.639333 16711 raft_consensus.cc:3054] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:15.646176 16533 catalog_manager.cc:5526] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.647234 16711 raft_consensus.cc:2463] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:15.650615 18050 raft_consensus.cc:1059] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:15.653319 18050 consensus_queue.cc:237] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.664254 17322 raft_consensus.cc:1270] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:15.666158 16787 raft_consensus.cc:1270] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:15.665714 16537 catalog_manager.cc:5526] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:15.669404 17993 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:15.670240 16783 consensus_queue.cc:237] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.671142 17941 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:15.674561 16634 raft_consensus.cc:1270] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:15.680380 17324 raft_consensus.cc:1270] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:15.685416 16634 raft_consensus.cc:1270] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:15.689993 18004 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:15.691397 17995 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:15.695686 18049 consensus_queue.cc:1035] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:15.697180 18050 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.704686 16783 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.710399 17324 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.718209 16533 catalog_manager.cc:5526] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:15.725649 16639 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.737339 17924 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.738893 17322 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.746311 16524 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet b1d0b3432b644821ba556f59a893d40a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:15.749840 16711 consensus_queue.cc:237] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:15.754282 16639 raft_consensus.cc:2949] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.761912 17321 raft_consensus.cc:1270] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:15.763214 17941 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:15.767478 16787 raft_consensus.cc:1270] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:15.769686 18050 consensus_queue.cc:1035] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:15.770321 16537 catalog_manager.cc:5526] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.772454 17993 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.773924 17324 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.788470 16783 raft_consensus.cc:2949] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.794973 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 2bdb0f016b8946c582e26ab6e02cb9b1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:15.805160 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet b1d0b3432b644821ba556f59a893d40a with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:15.813896 16537 catalog_manager.cc:5526] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:15.821272 18061 tablet_replica.cc:331] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:15.822788 18061 raft_consensus.cc:2238] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:15.823545 18061 raft_consensus.cc:2267] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:15.826705 18061 ts_tablet_manager.cc:1905] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:15.829856 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bdb0f016b8946c582e26ab6e02cb9b1 with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:15.833712 16632 raft_consensus.cc:1270] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:15.833920 18062 tablet_replica.cc:331] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:15.835117 18062 raft_consensus.cc:2238] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:15.835824 18062 raft_consensus.cc:2267] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:15.835750 17924 consensus_queue.cc:1035] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:15.839229 18062 ts_tablet_manager.cc:1905] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:15.842690 18061 ts_tablet_manager.cc:1918] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:15.843125 18061 log.cc:1198] T b1d0b3432b644821ba556f59a893d40a P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/b1d0b3432b644821ba556f59a893d40a
I20250112 20:58:15.844805 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet b1d0b3432b644821ba556f59a893d40a (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:15.855065 18062 ts_tablet_manager.cc:1918] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:15.855796 18062 log.cc:1198] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/2bdb0f016b8946c582e26ab6e02cb9b1
I20250112 20:58:15.858654 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 2bdb0f016b8946c582e26ab6e02cb9b1 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:15.861039 16711 raft_consensus.cc:1270] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:15.862191 17924 consensus_queue.cc:1035] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:15.881986 17324 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:15.883195 17924 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:15.892982 16639 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:15.894886 18049 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:16.026705 16783 raft_consensus.cc:1270] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:16.027772 18036 consensus_queue.cc:1035] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:16.035449 16708 raft_consensus.cc:1270] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:16.036908 18036 consensus_queue.cc:1035] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:16.177947 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:16.209591 16639 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:16.210033 16639 raft_consensus.cc:604] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:16.212049 16787 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:16.212481 16787 raft_consensus.cc:604] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:16.214429 16787 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:16.214805 16787 raft_consensus.cc:604] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:16.215643 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20250112 20:58:16.216688 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:16.217242 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:16.401120 17924 raft_consensus.cc:988] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:16.401648 18049 raft_consensus.cc:1076] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:16.402869 16708 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:58990
I20250112 20:58:16.403327 16708 raft_consensus.cc:491] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:16.403571 16708 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:16.407629 16708 raft_consensus.cc:513] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:16.408953 16708 leader_election.cc:290] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:16.409592 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:16.409775 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:16.410182 16639 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:16.410290 16787 raft_consensus.cc:3049] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:16.410583 16787 raft_consensus.cc:738] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:16.411160 16787 consensus_queue.cc:260] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:16.412202 16787 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:16.414970 16639 raft_consensus.cc:2463] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:16.415865 16678 leader_election.cc:304] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:16.416540 18050 raft_consensus.cc:2798] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:16.416790 16787 raft_consensus.cc:2463] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:16.416940 18050 raft_consensus.cc:695] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:16.417834 18050 consensus_queue.cc:237] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:16.426067 16537 catalog_manager.cc:5526] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:16.435544 17924 raft_consensus.cc:988] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:16.435972 18049 raft_consensus.cc:1076] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:16.437044 16639 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50052
I20250112 20:58:16.437418 16639 raft_consensus.cc:491] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:16.437649 16639 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:16.442070 16639 raft_consensus.cc:513] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:16.443332 16639 leader_election.cc:290] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 3 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:16.443897 16708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:16.444168 16787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:16.444482 16708 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:16.444728 16787 raft_consensus.cc:3049] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:58:16.446688 16787 raft_consensus.cc:738] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:16.447502 16787 consensus_queue.cc:260] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:16.448493 16787 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:16.450033 16708 raft_consensus.cc:2463] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 3.
I20250112 20:58:16.450888 16602 leader_election.cc:304] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:16.451426 17971 raft_consensus.cc:2798] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:16.451825 17971 raft_consensus.cc:695] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 3 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:16.452466 17971 consensus_queue.cc:237] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:16.452877 16787 raft_consensus.cc:2463] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 3.
I20250112 20:58:16.457991 18036 raft_consensus.cc:988] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:16.458441 17971 raft_consensus.cc:1076] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:16.464947 17324 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53250
I20250112 20:58:16.465436 17324 raft_consensus.cc:491] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:16.465762 17324 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:16.465894 16537 catalog_manager.cc:5526] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 2 to 3, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 3 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:16.472374 17324 raft_consensus.cc:513] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:16.474730 17324 leader_election.cc:290] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:16.475066 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:16.475600 16639 raft_consensus.cc:3049] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:58:16.475701 16708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4799def22f754c9881fd8245eb8f75b3" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:16.475926 16639 raft_consensus.cc:738] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:16.476215 16708 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:16.476563 16639 consensus_queue.cc:260] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:16.477764 16639 raft_consensus.cc:3054] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:16.481595 16708 raft_consensus.cc:2463] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 3.
I20250112 20:58:16.482421 17287 leader_election.cc:304] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters:
I20250112 20:58:16.483112 17967 raft_consensus.cc:2798] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:16.483453 16639 raft_consensus.cc:2463] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 3.
I20250112 20:58:16.483496 17967 raft_consensus.cc:695] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:16.484503 17967 consensus_queue.cc:237] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:16.493276 16537 catalog_manager.cc:5526] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 2 to 3, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 3 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:16.839569 16787 raft_consensus.cc:1270] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:16.840832 17971 consensus_queue.cc:1035] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:16.850760 16711 raft_consensus.cc:1270] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:16.852192 17968 consensus_queue.cc:1035] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:16.891785 16711 consensus_queue.cc:237] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:16.897527 16787 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20250112 20:58:16.898655 16678 consensus_peers.cc:487] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04890c40fe454c4a91d3d12e4104200a. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:16.898708 16632 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:16.899296 18084 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:16.900228 18050 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:58:16.906697 18050 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:16.908110 16785 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:16.916167 16639 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:16.922425 16632 consensus_queue.cc:237] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:16.931198 16711 raft_consensus.cc:1270] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:16.933578 16785 raft_consensus.cc:1270] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:16.933717 18036 consensus_queue.cc:1035] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:16.935382 18036 consensus_queue.cc:1035] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:16.928436 16537 catalog_manager.cc:5526] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:58:16.937217 16603 consensus_peers.cc:487] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b9eab84e9c44a4aae8715e7735721d8. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:16.951522 17971 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:16.953172 16711 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:16.960245 16785 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:16.966327 16632 consensus_queue.cc:237] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:16.976034 16787 raft_consensus.cc:1270] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:16.977784 16537 catalog_manager.cc:5526] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:16.980297 16785 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:16.981788 17971 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:16.983461 16711 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:16.983822 18084 consensus_queue.cc:1035] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:16.987596 16603 consensus_peers.cc:487] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1a1a7f345014fa7aff0f0cc0160789f. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:16.989859 17968 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:16.997632 16632 raft_consensus.cc:1270] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:16.997293 17968 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:16.999011 17995 consensus_queue.cc:1035] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:17.013728 16708 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:17.021944 16785 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:17.036079 16532 catalog_manager.cc:5526] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:17.069835 16632 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250112 20:58:17.071128 18079 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 20:58:17.073725 16708 consensus_queue.cc:237] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.075999 16711 raft_consensus.cc:1270] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250112 20:58:17.077535 17895 consensus_queue.cc:1035] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20250112 20:58:17.148277 16678 consensus_peers.cc:487] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 86b544e7e4fa4e419949c833428c406b. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:17.150754 16636 raft_consensus.cc:1270] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:17.150900 16787 raft_consensus.cc:1270] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:17.152737 18084 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:17.155100 17995 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:17.166749 17995 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:17.171113 16632 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:17.189143 16537 catalog_manager.cc:5526] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:17.191026 16785 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:17.250097 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:17.291664 16636 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:17.292213 16636 raft_consensus.cc:604] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:17.294191 16711 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:17.294701 16711 raft_consensus.cc:604] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Received request to transfer leadership to TS c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:17.295639 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:58:17.296906 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:17.297389 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:17.313412 18110 ts_tablet_manager.cc:927] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:17.320248 18110 tablet_copy_client.cc:323] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:17.321727 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 04890c40fe454c4a91d3d12e4104200a from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:17.322242 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 04890c40fe454c4a91d3d12e4104200a from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-04890c40fe454c4a91d3d12e4104200a
I20250112 20:58:17.332916 16724 tablet_copy_source_session.cc:215] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:17.337693 18110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04890c40fe454c4a91d3d12e4104200a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:17.353579 18110 tablet_copy_client.cc:806] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:17.354099 18110 tablet_copy_client.cc:670] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:17.357285 18110 tablet_copy_client.cc:538] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:17.364806 18110 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:17.385748 18110 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:17.386570 18110 tablet_bootstrap.cc:492] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:17.387203 18110 ts_tablet_manager.cc:1397] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.023s user 0.022s sys 0.000s
I20250112 20:58:17.389691 18110 raft_consensus.cc:357] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.390654 18110 raft_consensus.cc:738] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:17.391273 18110 consensus_queue.cc:260] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.393846 18110 ts_tablet_manager.cc:1428] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20250112 20:58:17.395802 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-04890c40fe454c4a91d3d12e4104200a received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:17.396221 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-04890c40fe454c4a91d3d12e4104200a on tablet 04890c40fe454c4a91d3d12e4104200a with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:17.476250 18110 ts_tablet_manager.cc:927] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:17.477841 18110 tablet_copy_client.cc:323] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:17.478986 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 1b9eab84e9c44a4aae8715e7735721d8 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:17.479437 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 1b9eab84e9c44a4aae8715e7735721d8 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-1b9eab84e9c44a4aae8715e7735721d8
I20250112 20:58:17.485597 16649 tablet_copy_source_session.cc:215] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:17.487654 18110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b9eab84e9c44a4aae8715e7735721d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:17.499078 17968 raft_consensus.cc:988] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:17.499611 18097 raft_consensus.cc:1076] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:17.501063 16714 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:59006
I20250112 20:58:17.501454 16714 raft_consensus.cc:491] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:17.501677 16714 raft_consensus.cc:3054] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:17.501943 18110 tablet_copy_client.cc:806] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:17.502384 18110 tablet_copy_client.cc:670] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:17.505298 18110 tablet_copy_client.cc:538] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:17.506716 16714 raft_consensus.cc:513] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.508404 16714 leader_election.cc:290] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:17.509007 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:17.509189 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1a1a7f345014fa7aff0f0cc0160789f" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:17.509625 16636 raft_consensus.cc:3049] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:17.509754 16783 raft_consensus.cc:3054] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:17.510004 16636 raft_consensus.cc:738] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:17.510731 16636 consensus_queue.cc:260] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.512056 16636 raft_consensus.cc:3054] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:17.512696 18110 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:17.514254 16783 raft_consensus.cc:2463] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:17.515708 16680 leader_election.cc:304] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:17.516393 18087 raft_consensus.cc:2798] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:17.516853 18087 raft_consensus.cc:695] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:17.517688 18087 consensus_queue.cc:237] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.522472 16636 raft_consensus.cc:2463] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:17.527949 16534 catalog_manager.cc:5526] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:17.532231 18110 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:17.533069 18110 tablet_bootstrap.cc:492] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:17.534118 18110 ts_tablet_manager.cc:1397] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.022s user 0.018s sys 0.004s
I20250112 20:58:17.536823 18110 raft_consensus.cc:357] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.537688 18110 raft_consensus.cc:738] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:17.538342 18110 consensus_queue.cc:260] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.539031 18050 raft_consensus.cc:988] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d: : Instructing follower c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:17.539508 18089 raft_consensus.cc:1076] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Signalling peer c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:17.541373 16783 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:39356
I20250112 20:58:17.541824 16783 raft_consensus.cc:491] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:17.542186 16783 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:17.545303 18110 ts_tablet_manager.cc:1428] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250112 20:58:17.547070 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-1b9eab84e9c44a4aae8715e7735721d8 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:17.547425 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-1b9eab84e9c44a4aae8715e7735721d8 on tablet 1b9eab84e9c44a4aae8715e7735721d8 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:17.558048 16783 raft_consensus.cc:513] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:17.560534 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:17.561105 16714 raft_consensus.cc:3049] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:58:17.561293 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b1a80e42504299a3970ea4437b8a13" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:17.561465 16714 raft_consensus.cc:738] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:17.561793 16636 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:17.562084 16714 consensus_queue.cc:260] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:17.563009 16714 raft_consensus.cc:3054] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:17.560082 16783 leader_election.cc:290] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 3 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:17.570639 16714 raft_consensus.cc:2463] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 3.
I20250112 20:58:17.572840 16751 leader_election.cc:304] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:17.573809 18098 raft_consensus.cc:2798] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:17.574237 18098 raft_consensus.cc:695] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 3 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:17.574916 18098 consensus_queue.cc:237] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:17.575315 16636 raft_consensus.cc:2463] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 3.
I20250112 20:58:17.586449 16534 catalog_manager.cc:5526] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 2 to 3, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 3 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:17.708117 18110 ts_tablet_manager.cc:927] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:17.710023 18110 tablet_copy_client.cc:323] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:17.711483 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 86b544e7e4fa4e419949c833428c406b from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:17.711992 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 86b544e7e4fa4e419949c833428c406b from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-86b544e7e4fa4e419949c833428c406b
I20250112 20:58:17.718297 16724 tablet_copy_source_session.cc:215] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:17.720957 18110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86b544e7e4fa4e419949c833428c406b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:17.733700 18110 tablet_copy_client.cc:806] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:17.734208 18110 tablet_copy_client.cc:670] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:17.737183 18110 tablet_copy_client.cc:538] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:17.744045 18110 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:17.760888 18110 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:17.761508 18110 tablet_bootstrap.cc:492] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:17.761952 18110 ts_tablet_manager.cc:1397] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.004s
I20250112 20:58:17.763733 18110 raft_consensus.cc:357] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.764353 18110 raft_consensus.cc:738] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:17.764813 18110 consensus_queue.cc:260] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:17.766863 18110 ts_tablet_manager.cc:1428] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250112 20:58:17.770604 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-86b544e7e4fa4e419949c833428c406b received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:17.770979 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-86b544e7e4fa4e419949c833428c406b on tablet 86b544e7e4fa4e419949c833428c406b with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:17.907550 16636 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:17.909142 17995 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:17.930145 17323 raft_consensus.cc:1212] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
W20250112 20:58:17.937582 16678 consensus_peers.cc:487] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1a1a7f345014fa7aff0f0cc0160789f. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:17.939447 16783 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:17.941798 18087 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:18.031666 17323 raft_consensus.cc:1212] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:18.055778 16636 raft_consensus.cc:1270] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:18.056795 18098 consensus_queue.cc:1035] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:18.067746 16714 raft_consensus.cc:1270] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:18.068957 18098 consensus_queue.cc:1035] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:58:18.135243 17323 raft_consensus.cc:1212] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:18.320375 18125 ts_tablet_manager.cc:927] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:18.322227 18125 tablet_copy_client.cc:323] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:18.323359 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet c1a1a7f345014fa7aff0f0cc0160789f from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:18.323804 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet c1a1a7f345014fa7aff0f0cc0160789f from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-c1a1a7f345014fa7aff0f0cc0160789f
I20250112 20:58:18.330394 16724 tablet_copy_source_session.cc:215] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:18.332661 18125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a1a7f345014fa7aff0f0cc0160789f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:18.343726 18125 tablet_copy_client.cc:806] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:18.344292 18125 tablet_copy_client.cc:670] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:18.347489 18125 tablet_copy_client.cc:538] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:18.351258 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:18.354336 18125 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:18.373163 18089 raft_consensus.cc:1059] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:18.375370 18089 consensus_queue.cc:237] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:18.389993 16783 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:18.390151 16632 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:18.391619 17323 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:18.392994 18087 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:18.394773 18050 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:18.395788 17995 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:18.396800 18125 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:18.397636 18125 tablet_bootstrap.cc:492] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:18.398247 18125 ts_tablet_manager.cc:1397] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.044s user 0.017s sys 0.011s
I20250112 20:58:18.401763 18125 raft_consensus.cc:357] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:18.402575 18125 raft_consensus.cc:738] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:18.403201 18125 consensus_queue.cc:260] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:18.412734 18125 ts_tablet_manager.cc:1428] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250112 20:58:18.414132 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-c1a1a7f345014fa7aff0f0cc0160789f received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:18.414505 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-c1a1a7f345014fa7aff0f0cc0160789f on tablet c1a1a7f345014fa7aff0f0cc0160789f with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:18.422479 18050 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.429831 17323 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.430763 16783 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.433526 16708 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "03cbd278697d4f6780211b21028f6d18"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:18.434080 16708 raft_consensus.cc:604] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:18.440522 16784 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:18.441080 16784 raft_consensus.cc:604] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:18.447010 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:58:18.446897 16632 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.448547 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:18.449170 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:18.451539 16534 catalog_manager.cc:5526] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.473342 18098 raft_consensus.cc:988] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:18.473803 18098 raft_consensus.cc:1076] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:18.478921 16711 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:58990
I20250112 20:58:18.479382 16711 raft_consensus.cc:491] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:18.479671 16711 raft_consensus.cc:3054] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:18.486536 16711 raft_consensus.cc:513] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:18.497172 16632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:18.497656 16632 raft_consensus.cc:3054] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:18.499488 16708 consensus_queue.cc:237] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:18.503481 16632 raft_consensus.cc:2463] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:18.505241 16678 leader_election.cc:304] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:18.506505 17995 raft_consensus.cc:2798] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:18.532150 16711 leader_election.cc:290] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:18.533468 16783 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:18.505443 17323 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:18.541533 17995 raft_consensus.cc:695] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:18.542541 17995 consensus_queue.cc:237] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:18.556268 18050 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:18.557632 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3206c0399f4057928460c7a6f04912" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:18.560822 16783 raft_consensus.cc:3049] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:18.563546 16783 raft_consensus.cc:738] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:18.564965 16783 consensus_queue.cc:260] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:18.571065 16783 raft_consensus.cc:3054] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:18.587450 18089 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250112 20:58:18.591576 18087 raft_consensus.cc:1059] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
W20250112 20:58:18.595420 16680 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.131:38713, user_credentials={real_user=slave}} blocked reactor thread for 50347.7us
I20250112 20:58:18.597450 17971 raft_consensus.cc:1059] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:18.599890 17971 consensus_queue.cc:237] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:18.601994 18087 consensus_queue.cc:237] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:18.605388 18089 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.611490 16787 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.613579 17323 raft_consensus.cc:2949] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.618844 17322 raft_consensus.cc:1270] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:18.620566 18097 consensus_queue.cc:1035] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20250112 20:58:18.621687 16784 raft_consensus.cc:1270] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:18.625193 18120 consensus_queue.cc:1035] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:18.633255 16711 raft_consensus.cc:1270] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:18.634639 17971 consensus_queue.cc:1035] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:18.634784 16534 catalog_manager.cc:5526] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 4 to 5, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.636958 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 04890c40fe454c4a91d3d12e4104200a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:18.648574 16532 catalog_manager.cc:5526] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 1 to 2, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:18.655954 16783 raft_consensus.cc:2463] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 2.
I20250112 20:58:18.669366 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 04890c40fe454c4a91d3d12e4104200a with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:18.675292 17322 raft_consensus.cc:1270] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:18.676379 16783 raft_consensus.cc:1270] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:18.681397 18050 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:18.682200 18089 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:18.683367 18144 tablet_replica.cc:331] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:18.684033 17971 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.685273 18144 raft_consensus.cc:2238] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:18.685983 18144 raft_consensus.cc:2267] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:18.688225 16632 raft_consensus.cc:1270] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:18.689383 18144 ts_tablet_manager.cc:1905] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:18.691574 16711 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.697126 16784 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.707639 18089 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:18.714531 17324 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.733531 16537 catalog_manager.cc:5526] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.736891 18144 ts_tablet_manager.cc:1918] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250112 20:58:18.737401 18144 log.cc:1198] T 04890c40fe454c4a91d3d12e4104200a P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/04890c40fe454c4a91d3d12e4104200a
I20250112 20:58:18.747082 18135 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.748205 17322 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.758635 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 04890c40fe454c4a91d3d12e4104200a (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:18.765065 16532 catalog_manager.cc:5526] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.770529 16632 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.772120 16636 consensus_queue.cc:237] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:18.773252 16783 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } attrs { replace: true } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.777330 17322 raft_consensus.cc:1270] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:18.779927 17971 consensus_queue.cc:1035] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:18.789512 16785 raft_consensus.cc:1270] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:18.791733 18120 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.794517 17971 consensus_queue.cc:1035] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:18.798579 16714 consensus_queue.cc:237] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:18.795890 17322 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.801286 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 1b9eab84e9c44a4aae8715e7735721d8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:18.806793 17323 raft_consensus.cc:1270] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:18.809504 16784 raft_consensus.cc:1270] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:18.810871 18136 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:18.810005 16785 raft_consensus.cc:2949] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.818696 18136 consensus_queue.cc:1035] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:18.816370 16537 catalog_manager.cc:5526] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.823098 17322 raft_consensus.cc:1212] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20250112 20:58:18.829583 18087 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.847944 16784 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.849915 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b9eab84e9c44a4aae8715e7735721d8 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:18.856760 17323 raft_consensus.cc:2949] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.868417 18135 raft_consensus.cc:988] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d: : Instructing follower 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:18.868582 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 86b544e7e4fa4e419949c833428c406b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:18.868912 18089 raft_consensus.cc:1076] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer 80770068c3a34f918765f54dbe3a94eb to start an election
I20250112 20:58:18.870271 16636 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "03cbd278697d4f6780211b21028f6d18"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
from {username='slave'} at 127.0.0.1:50066
I20250112 20:58:18.870759 16636 raft_consensus.cc:491] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:18.871070 16636 raft_consensus.cc:3054] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:18.874579 18149 tablet_replica.cc:331] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:18.875737 18149 raft_consensus.cc:2238] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:18.877818 18149 raft_consensus.cc:2267] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:18.874984 16537 catalog_manager.cc:5526] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 3 to 4, VOTER 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) evicted. New cstate: current_term: 1 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:18.879863 16636 raft_consensus.cc:513] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:18.881765 18149 ts_tablet_manager.cc:1905] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:18.882422 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03cbd278697d4f6780211b21028f6d18" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:18.882983 16714 raft_consensus.cc:3049] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:18.883256 16714 raft_consensus.cc:738] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:18.883898 16714 consensus_queue.cc:260] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:18.885303 16714 raft_consensus.cc:3054] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:18.888185 16625 tablet_service.cc:1514] Processing DeleteTablet for tablet 86b544e7e4fa4e419949c833428c406b with delete_type TABLET_DATA_TOMBSTONED (TS 80770068c3a34f918765f54dbe3a94eb not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33112
I20250112 20:58:18.889113 18144 tablet_replica.cc:331] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:18.889181 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03cbd278697d4f6780211b21028f6d18" candidate_uuid: "80770068c3a34f918765f54dbe3a94eb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:18.889807 16785 raft_consensus.cc:3054] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:18.890197 18144 raft_consensus.cc:2238] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:18.890980 18144 raft_consensus.cc:2267] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:18.894615 18144 ts_tablet_manager.cc:1905] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:18.900278 16785 raft_consensus.cc:2463] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:18.901499 16605 leader_election.cc:304] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:18.902181 17971 raft_consensus.cc:2798] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:18.902568 17971 raft_consensus.cc:695] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Becoming Leader. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:18.903270 17971 consensus_queue.cc:237] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:18.904301 16714 raft_consensus.cc:2463] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80770068c3a34f918765f54dbe3a94eb in term 2.
I20250112 20:58:18.900331 18149 ts_tablet_manager.cc:1918] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:18.905457 18149 log.cc:1198] T 1b9eab84e9c44a4aae8715e7735721d8 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/1b9eab84e9c44a4aae8715e7735721d8
I20250112 20:58:18.900411 16636 leader_election.cc:290] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [CANDIDATE]: Term 2 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:18.907639 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 1b9eab84e9c44a4aae8715e7735721d8 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:18.914798 18144 ts_tablet_manager.cc:1918] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:18.915179 18144 log.cc:1198] T 86b544e7e4fa4e419949c833428c406b P 80770068c3a34f918765f54dbe3a94eb: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/wals/86b544e7e4fa4e419949c833428c406b
I20250112 20:58:18.916392 16522 catalog_manager.cc:4872] TS 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): tablet 86b544e7e4fa4e419949c833428c406b (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:18.929852 16532 catalog_manager.cc:5526] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 80770068c3a34f918765f54dbe3a94eb (127.15.23.129). New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:18.973618 16636 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:18.973845 16784 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:18.975728 18089 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:18.976464 18087 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:19.200613 18087 raft_consensus.cc:1059] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:19.202244 18087 consensus_queue.cc:237] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:19.207010 16784 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:19.207656 16632 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:19.208003 17322 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:19.208755 18089 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:19.209601 18050 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:19.210897 18135 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:19.228716 18050 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.229910 16784 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.234115 16632 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.240999 17323 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.246194 16533 catalog_manager.cc:5526] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 2 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.261968 16714 consensus_queue.cc:237] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:19.271037 17322 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:19.272619 18135 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:19.279568 18089 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.280778 16632 raft_consensus.cc:1270] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:19.282081 18087 consensus_queue.cc:1035] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:19.281841 17323 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.285859 16636 raft_consensus.cc:2949] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.289451 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet c1a1a7f345014fa7aff0f0cc0160789f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:19.298841 16533 catalog_manager.cc:5526] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:19.309476 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet c1a1a7f345014fa7aff0f0cc0160789f with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:19.312197 18160 tablet_replica.cc:331] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:19.314268 18160 raft_consensus.cc:2238] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:19.315008 18160 raft_consensus.cc:2267] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:19.317886 18160 ts_tablet_manager.cc:1905] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:19.329012 18160 ts_tablet_manager.cc:1918] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250112 20:58:19.329387 18160 log.cc:1198] T c1a1a7f345014fa7aff0f0cc0160789f P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/c1a1a7f345014fa7aff0f0cc0160789f
I20250112 20:58:19.330507 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet c1a1a7f345014fa7aff0f0cc0160789f (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:19.367575 16784 raft_consensus.cc:1270] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:19.368587 17971 consensus_queue.cc:1035] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:19.385365 16714 raft_consensus.cc:1270] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:19.386384 17971 consensus_queue.cc:1035] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:19.640820 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:19.678182 16711 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:19.678613 16711 raft_consensus.cc:604] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:19.680478 16636 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40"
dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
mode: GRACEFUL
new_leader_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:45962
I20250112 20:58:19.680919 16636 raft_consensus.cc:604] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 3 LEADER]: Received request to transfer leadership to TS 018d03f043b542b9b63491a77523968d
I20250112 20:58:19.681733 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:58:19.682663 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:19.683041 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:19.770469 18089 raft_consensus.cc:988] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:19.770855 18087 raft_consensus.cc:1076] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:19.772078 17323 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53256
I20250112 20:58:19.772495 17323 raft_consensus.cc:491] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:19.772763 17323 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:19.776551 17323 raft_consensus.cc:513] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:19.777760 17323 leader_election.cc:290] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 election: Requested vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:19.778457 16711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:19.778580 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04890c40fe454c4a91d3d12e4104200a" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:19.778925 16711 raft_consensus.cc:3049] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:58:19.779090 16785 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:19.779201 16711 raft_consensus.cc:738] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:19.779948 16711 consensus_queue.cc:260] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:19.781106 16711 raft_consensus.cc:3054] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:19.783164 16785 raft_consensus.cc:2463] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 3.
I20250112 20:58:19.784057 17290 leader_election.cc:304] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:19.784632 18107 raft_consensus.cc:2798] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:19.785012 18107 raft_consensus.cc:695] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:19.785218 16711 raft_consensus.cc:2463] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 3.
I20250112 20:58:19.785916 18107 consensus_queue.cc:237] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:19.795907 16537 catalog_manager.cc:5526] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 2 to 3, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 3 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:20.079403 17971 raft_consensus.cc:988] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb: : Instructing follower 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:20.079803 18161 raft_consensus.cc:1076] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 3 LEADER]: Signalling peer 018d03f043b542b9b63491a77523968d to start an election
I20250112 20:58:20.081028 16711 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40"
dest_uuid: "018d03f043b542b9b63491a77523968d"
from {username='slave'} at 127.0.0.1:59006
I20250112 20:58:20.081449 16711 raft_consensus.cc:491] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:20.081705 16711 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:58:20.086930 16711 raft_consensus.cc:513] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:20.088615 16711 leader_election.cc:290] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 4 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:20.089358 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:20.089900 16783 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:58:20.092386 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab6f0844afa43d8b7af8e3705d09c40" candidate_uuid: "018d03f043b542b9b63491a77523968d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:20.092938 16636 raft_consensus.cc:3049] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 3 LEADER]: Stepping down as leader of term 3
I20250112 20:58:20.093204 16636 raft_consensus.cc:738] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 80770068c3a34f918765f54dbe3a94eb, State: Running, Role: LEADER
I20250112 20:58:20.093940 16636 consensus_queue.cc:260] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:20.094489 16783 raft_consensus.cc:2463] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 4.
I20250112 20:58:20.095090 16636 raft_consensus.cc:3054] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:58:20.095418 16680 leader_election.cc:304] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:20.096400 18087 raft_consensus.cc:2798] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 4 FOLLOWER]: Leader election won for term 4
I20250112 20:58:20.096822 18087 raft_consensus.cc:695] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 4 LEADER]: Becoming Leader. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:20.097890 18087 consensus_queue.cc:237] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:20.099550 16636 raft_consensus.cc:2463] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 018d03f043b542b9b63491a77523968d in term 4.
I20250112 20:58:20.105746 16537 catalog_manager.cc:5526] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d reported cstate change: term changed from 3 to 4, leader changed from 80770068c3a34f918765f54dbe3a94eb (127.15.23.129) to 018d03f043b542b9b63491a77523968d (127.15.23.130). New cstate: current_term: 4 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:20.268337 16783 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250112 20:58:20.269469 17895 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 20:58:20.280905 16711 raft_consensus.cc:1270] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250112 20:58:20.284709 18107 consensus_queue.cc:1035] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 20:58:20.365789 16636 consensus_queue.cc:237] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:20.371096 16785 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20250112 20:58:20.371170 16603 consensus_peers.cc:487] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 680c875f5d264744aef97e8dd9c304b9. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:20.371196 16714 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:20.372684 18120 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:20.373557 18161 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:20.388618 18161 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.389820 16714 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.393394 16785 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.405679 16714 consensus_queue.cc:237] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:20.406086 16537 catalog_manager.cc:5526] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250112 20:58:20.417626 16678 consensus_peers.cc:487] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b3206c0399f4057928460c7a6f04912. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:20.425242 16785 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:20.425335 16636 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:20.426774 18089 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:20.428237 18089 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:20.435281 18087 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.436141 16636 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.439507 16785 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.449044 16632 consensus_queue.cc:237] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:20.453588 16537 catalog_manager.cc:5526] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
W20250112 20:58:20.463343 16603 consensus_peers.cc:487] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 963bd285f345421581941f10dbe8b90c. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:20.466498 16785 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:20.466614 16714 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20250112 20:58:20.469301 18161 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:20.470166 18120 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:20.477839 18120 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.478921 16783 raft_consensus.cc:1270] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 4 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:58:20.480172 18087 consensus_queue.cc:1035] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:20.480408 16714 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.509163 16632 raft_consensus.cc:1270] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 4 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20250112 20:58:20.511083 18089 consensus_queue.cc:1035] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:20.514629 16532 catalog_manager.cc:5526] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d reported cstate change: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.534969 16785 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.537451 16632 consensus_queue.cc:237] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:20.543488 16783 raft_consensus.cc:1270] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250112 20:58:20.543829 16603 consensus_peers.cc:487] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb -> Peer 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633): Couldn't send request to peer 1b6f6e362d6c451d865ec54e6ce5dfe1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f021c2fa4d3143ff899283a33cb24538. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:20.544509 18120 consensus_queue.cc:1035] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:20.545605 16714 raft_consensus.cc:1270] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250112 20:58:20.546758 18120 consensus_queue.cc:1035] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:20.556126 18185 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.568094 16714 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.566823 16783 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } } }
I20250112 20:58:20.580526 16532 catalog_manager.cc:5526] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index -1 to 2, NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) added. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:20.715130 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:20.763968 16714 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39"
dest_uuid: "018d03f043b542b9b63491a77523968d"
mode: GRACEFUL
new_leader_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:38076
I20250112 20:58:20.764420 16714 raft_consensus.cc:604] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Received request to transfer leadership to TS c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:20.765363 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250112 20:58:20.766764 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:20.767277 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:20.773489 18192 ts_tablet_manager.cc:927] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:20.803157 18192 tablet_copy_client.cc:323] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:20.805752 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 680c875f5d264744aef97e8dd9c304b9 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:20.806198 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 680c875f5d264744aef97e8dd9c304b9 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-680c875f5d264744aef97e8dd9c304b9
I20250112 20:58:20.812772 16649 tablet_copy_source_session.cc:215] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:20.817762 18192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 680c875f5d264744aef97e8dd9c304b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:20.833407 18192 tablet_copy_client.cc:806] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:20.833918 18192 tablet_copy_client.cc:670] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:20.836988 18192 tablet_copy_client.cc:538] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:20.844791 18192 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:20.864256 18192 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:20.864915 18192 tablet_bootstrap.cc:492] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:20.865404 18192 ts_tablet_manager.cc:1397] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.015s
I20250112 20:58:20.867317 18192 raft_consensus.cc:357] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:20.867911 18192 raft_consensus.cc:738] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:20.868551 18192 consensus_queue.cc:260] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:20.871125 18192 ts_tablet_manager.cc:1428] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20250112 20:58:20.872982 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-680c875f5d264744aef97e8dd9c304b9 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:20.873374 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-680c875f5d264744aef97e8dd9c304b9 on tablet 680c875f5d264744aef97e8dd9c304b9 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:21.005028 18192 ts_tablet_manager.cc:927] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:21.006469 18192 tablet_copy_client.cc:323] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:21.007932 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet 963bd285f345421581941f10dbe8b90c from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:21.008410 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet 963bd285f345421581941f10dbe8b90c from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-963bd285f345421581941f10dbe8b90c
I20250112 20:58:21.014739 16649 tablet_copy_source_session.cc:215] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:21.017266 18192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 963bd285f345421581941f10dbe8b90c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:21.030037 18196 ts_tablet_manager.cc:927] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:21.032380 18192 tablet_copy_client.cc:806] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:21.033444 18192 tablet_copy_client.cc:670] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:21.034443 18196 tablet_copy_client.cc:323] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.130:38087
I20250112 20:58:21.035975 16724 tablet_copy_service.cc:140] P 018d03f043b542b9b63491a77523968d: Received BeginTabletCopySession request for tablet 5b3206c0399f4057928460c7a6f04912 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:48112)
I20250112 20:58:21.036336 16724 tablet_copy_service.cc:161] P 018d03f043b542b9b63491a77523968d: Beginning new tablet copy session on tablet 5b3206c0399f4057928460c7a6f04912 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:48112: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-5b3206c0399f4057928460c7a6f04912
I20250112 20:58:21.041608 18192 tablet_copy_client.cc:538] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:21.042002 16724 tablet_copy_source_session.cc:215] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:21.044298 18196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b3206c0399f4057928460c7a6f04912. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:21.050014 18192 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:21.070163 18198 raft_consensus.cc:988] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d: : Instructing follower c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:21.070722 18087 raft_consensus.cc:1076] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Signalling peer c29b243acf2b4f248606563fc96efc7d to start an election
I20250112 20:58:21.072068 16783 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
from {username='slave'} at 127.0.0.1:39356
I20250112 20:58:21.074175 16783 raft_consensus.cc:491] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:21.074501 16783 raft_consensus.cc:3054] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:21.078181 18196 tablet_copy_client.cc:806] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:21.078665 18196 tablet_copy_client.cc:670] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:21.081048 16783 raft_consensus.cc:513] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:21.082502 16783 leader_election.cc:290] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:21.083633 16714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "018d03f043b542b9b63491a77523968d"
I20250112 20:58:21.083880 18196 tablet_copy_client.cc:538] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:21.083809 16639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0f7e20973149ed977de7cdd902fc39" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:21.084170 16714 raft_consensus.cc:3049] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:21.084669 16639 raft_consensus.cc:3054] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:21.086252 16714 raft_consensus.cc:738] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 018d03f043b542b9b63491a77523968d, State: Running, Role: LEADER
I20250112 20:58:21.087075 16714 consensus_queue.cc:260] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:21.089316 16714 raft_consensus.cc:3054] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:21.091500 18196 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:21.092885 16639 raft_consensus.cc:2463] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:21.095013 16752 leader_election.cc:304] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d; no voters:
I20250112 20:58:21.095890 18165 raft_consensus.cc:2798] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:21.096316 18165 raft_consensus.cc:695] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Becoming Leader. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:21.097088 18165 consensus_queue.cc:237] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } }
I20250112 20:58:21.097904 18192 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:21.098691 18192 tablet_bootstrap.cc:492] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:21.099316 18192 ts_tablet_manager.cc:1397] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.049s user 0.016s sys 0.008s
I20250112 20:58:21.102123 18192 raft_consensus.cc:357] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:21.102896 18192 raft_consensus.cc:738] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:21.103475 18192 consensus_queue.cc:260] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:21.105341 18192 ts_tablet_manager.cc:1428] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:58:21.108387 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-963bd285f345421581941f10dbe8b90c received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:21.108753 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-963bd285f345421581941f10dbe8b90c on tablet 963bd285f345421581941f10dbe8b90c with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:21.113860 18196 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:21.114627 18196 tablet_bootstrap.cc:492] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:21.115196 18196 ts_tablet_manager.cc:1397] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.024s user 0.018s sys 0.003s
I20250112 20:58:21.115968 16714 raft_consensus.cc:2463] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c29b243acf2b4f248606563fc96efc7d in term 2.
I20250112 20:58:21.117688 18196 raft_consensus.cc:357] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:21.118536 18196 raft_consensus.cc:738] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:21.119189 18196 consensus_queue.cc:260] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:21.121130 18196 ts_tablet_manager.cc:1428] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20250112 20:58:21.123096 16724 tablet_copy_service.cc:342] P 018d03f043b542b9b63491a77523968d: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-5b3206c0399f4057928460c7a6f04912 received from {username='slave'} at 127.0.0.1:48112
I20250112 20:58:21.123472 16724 tablet_copy_service.cc:434] P 018d03f043b542b9b63491a77523968d: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-5b3206c0399f4057928460c7a6f04912 on tablet 5b3206c0399f4057928460c7a6f04912 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:21.144846 16533 catalog_manager.cc:5526] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d reported cstate change: term changed from 1 to 2, leader changed from 018d03f043b542b9b63491a77523968d (127.15.23.130) to c29b243acf2b4f248606563fc96efc7d (127.15.23.131). New cstate: current_term: 2 leader_uuid: "c29b243acf2b4f248606563fc96efc7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:21.163691 18196 ts_tablet_manager.cc:927] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Initiating tablet copy from peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463)
I20250112 20:58:21.165855 18196 tablet_copy_client.cc:323] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Beginning tablet copy session from remote peer at address 127.15.23.129:41463
I20250112 20:58:21.167152 16649 tablet_copy_service.cc:140] P 80770068c3a34f918765f54dbe3a94eb: Received BeginTabletCopySession request for tablet f021c2fa4d3143ff899283a33cb24538 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 ({username='slave'} at 127.0.0.1:36254)
I20250112 20:58:21.167582 16649 tablet_copy_service.cc:161] P 80770068c3a34f918765f54dbe3a94eb: Beginning new tablet copy session on tablet f021c2fa4d3143ff899283a33cb24538 from peer 1b6f6e362d6c451d865ec54e6ce5dfe1 at {username='slave'} at 127.0.0.1:36254: session id = 1b6f6e362d6c451d865ec54e6ce5dfe1-f021c2fa4d3143ff899283a33cb24538
I20250112 20:58:21.173720 16649 tablet_copy_source_session.cc:215] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb: Tablet Copy: opened 0 blocks and 1 log segments
I20250112 20:58:21.176326 18196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f021c2fa4d3143ff899283a33cb24538. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:21.188262 18196 tablet_copy_client.cc:806] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 0 data blocks...
I20250112 20:58:21.188764 18196 tablet_copy_client.cc:670] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Starting download of 1 WAL segments...
I20250112 20:58:21.191424 18196 tablet_copy_client.cc:538] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250112 20:58:21.199122 18196 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap starting.
I20250112 20:58:21.216259 18196 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:21.216995 18196 tablet_bootstrap.cc:492] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Bootstrap complete.
I20250112 20:58:21.217541 18196 ts_tablet_manager.cc:1397] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20250112 20:58:21.220645 18196 raft_consensus.cc:357] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:21.221452 18196 raft_consensus.cc:738] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Initialized, Role: LEARNER
I20250112 20:58:21.222040 18196 consensus_queue.cc:260] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: NON_VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: true } }
I20250112 20:58:21.224320 18196 ts_tablet_manager.cc:1428] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:58:21.226792 16649 tablet_copy_service.cc:342] P 80770068c3a34f918765f54dbe3a94eb: Request end of tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-f021c2fa4d3143ff899283a33cb24538 received from {username='slave'} at 127.0.0.1:36254
I20250112 20:58:21.227145 16649 tablet_copy_service.cc:434] P 80770068c3a34f918765f54dbe3a94eb: ending tablet copy session 1b6f6e362d6c451d865ec54e6ce5dfe1-f021c2fa4d3143ff899283a33cb24538 on tablet f021c2fa4d3143ff899283a33cb24538 with peer 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:21.358510 17322 raft_consensus.cc:1212] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250112 20:58:21.415294 17322 raft_consensus.cc:1212] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250112 20:58:21.550376 16711 raft_consensus.cc:1270] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:21.551950 18202 consensus_queue.cc:1035] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:21.569319 16639 raft_consensus.cc:1270] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer c29b243acf2b4f248606563fc96efc7d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:21.573575 18123 consensus_queue.cc:1035] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:21.608507 17322 raft_consensus.cc:1212] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20250112 20:58:21.703074 17322 raft_consensus.cc:1212] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20250112 20:58:21.824751 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:21.857052 18161 raft_consensus.cc:1059] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:21.859234 18161 consensus_queue.cc:237] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:21.870524 17322 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:21.871151 16785 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774"
dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
mode: GRACEFUL
new_leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:57138
I20250112 20:58:21.871309 16714 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:21.871752 16785 raft_consensus.cc:604] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Received request to transfer leadership to TS 1b6f6e362d6c451d865ec54e6ce5dfe1
I20250112 20:58:21.871912 18161 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:58:21.872843 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20250112 20:58:21.873373 18185 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:21.874624 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:21.875029 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:21.877290 16785 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:21.879681 18185 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:21.887580 18185 raft_consensus.cc:1059] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:21.889892 18185 consensus_queue.cc:237] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:21.892241 18208 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:21.893186 16711 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:21.908965 17323 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:21.922545 16532 catalog_manager.cc:5526] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:21.933971 17323 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:21.935555 16714 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:21.935914 16783 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:21.937992 18208 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:58:21.940989 18219 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:21.943771 18222 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20250112 20:58:21.984308 16784 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.013435 18202 raft_consensus.cc:988] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d: : Instructing follower 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:22.014034 18202 raft_consensus.cc:1076] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Signalling peer 1b6f6e362d6c451d865ec54e6ce5dfe1 to start an election
I20250112 20:58:22.017714 17322 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774"
dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1"
from {username='slave'} at 127.0.0.1:53246
I20250112 20:58:22.018340 17322 raft_consensus.cc:491] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:22.018729 17322 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:22.021443 18161 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.033164 17322 raft_consensus.cc:513] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.036047 16785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c29b243acf2b4f248606563fc96efc7d"
I20250112 20:58:22.036604 16785 raft_consensus.cc:3049] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:58:22.036882 16785 raft_consensus.cc:738] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c29b243acf2b4f248606563fc96efc7d, State: Running, Role: LEADER
I20250112 20:58:22.038197 17324 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.037566 16785 consensus_queue.cc:260] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.042649 17322 leader_election.cc:290] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 election: Requested vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:22.043143 16785 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:22.045511 16636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807445e11d91471bbfa7a55fcb99f774" candidate_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "80770068c3a34f918765f54dbe3a94eb"
I20250112 20:58:22.052060 16636 raft_consensus.cc:3054] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:22.047991 16711 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.076825 16783 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.080225 16639 consensus_queue.cc:237] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.090759 16636 raft_consensus.cc:2463] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 3.
I20250112 20:58:22.093456 17288 leader_election.cc:304] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, 80770068c3a34f918765f54dbe3a94eb; no voters:
I20250112 20:58:22.094287 18229 raft_consensus.cc:2798] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:22.094322 17322 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:22.094839 18229 raft_consensus.cc:695] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 LEADER]: Becoming Leader. State: Replica: 1b6f6e362d6c451d865ec54e6ce5dfe1, State: Running, Role: LEADER
I20250112 20:58:22.095697 18229 consensus_queue.cc:237] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.096042 18219 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:22.098016 16785 raft_consensus.cc:2463] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6f6e362d6c451d865ec54e6ce5dfe1 in term 3.
I20250112 20:58:22.106449 18226 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.109328 16708 raft_consensus.cc:1270] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:22.111987 17322 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.125178 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 963bd285f345421581941f10dbe8b90c with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:22.124377 18226 consensus_queue.cc:1035] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20250112 20:58:22.133422 16708 raft_consensus.cc:2949] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.131768 16533 catalog_manager.cc:5526] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:22.149942 16636 consensus_queue.cc:237] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.155498 17322 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:22.156535 18185 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:22.159581 16534 catalog_manager.cc:5526] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: term changed from 2 to 3, leader changed from c29b243acf2b4f248606563fc96efc7d (127.15.23.131) to 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132). New cstate: current_term: 3 leader_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20250112 20:58:22.162771 16785 raft_consensus.cc:1270] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:22.162890 16534 catalog_manager.cc:5526] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.165663 18185 consensus_queue.cc:1035] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:22.169373 18198 raft_consensus.cc:1059] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:22.171516 18198 consensus_queue.cc:237] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.179760 18208 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.180912 16783 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.189230 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 680c875f5d264744aef97e8dd9c304b9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:22.191032 17322 raft_consensus.cc:2949] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.195281 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 963bd285f345421581941f10dbe8b90c with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:22.199512 17324 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEARNER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:22.200163 16783 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:22.202697 16636 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20250112 20:58:22.204213 18089 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:22.208201 18089 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:22.214375 18089 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:22.222427 16533 catalog_manager.cc:5526] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 4 to 5, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 2 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.226624 17324 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.230192 18234 tablet_replica.cc:331] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:22.226850 18089 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.231333 18234 raft_consensus.cc:2238] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:22.232086 18234 raft_consensus.cc:2267] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:22.235265 18234 ts_tablet_manager.cc:1905] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:22.241736 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet 680c875f5d264744aef97e8dd9c304b9 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:22.259953 18234 ts_tablet_manager.cc:1918] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250112 20:58:22.260638 18234 log.cc:1198] T 963bd285f345421581941f10dbe8b90c P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/963bd285f345421581941f10dbe8b90c
I20250112 20:58:22.262391 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet 963bd285f345421581941f10dbe8b90c (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:22.273038 18241 tablet_replica.cc:331] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:22.282368 16785 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.286340 16636 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.295776 16533 catalog_manager.cc:5526] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 3 to 4, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } attrs { replace: true } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.297462 18241 raft_consensus.cc:2238] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:22.298382 18241 raft_consensus.cc:2267] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:22.301787 18241 ts_tablet_manager.cc:1905] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:22.313712 18208 raft_consensus.cc:1059] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb: attempting to promote NON_VOTER 1b6f6e362d6c451d865ec54e6ce5dfe1 to VOTER
I20250112 20:58:22.316027 18208 consensus_queue.cc:237] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.334245 16711 consensus_queue.cc:237] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.336995 18241 ts_tablet_manager.cc:1918] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250112 20:58:22.337509 18241 log.cc:1198] T 680c875f5d264744aef97e8dd9c304b9 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/680c875f5d264744aef97e8dd9c304b9
I20250112 20:58:22.341585 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet 680c875f5d264744aef97e8dd9c304b9 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:22.344233 17322 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:22.346665 16708 raft_consensus.cc:1270] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:22.344233 16632 raft_consensus.cc:1270] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Refusing update from remote peer 018d03f043b542b9b63491a77523968d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20250112 20:58:22.347954 18185 consensus_queue.cc:1035] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:22.349851 18089 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:22.351338 18089 consensus_queue.cc:1035] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20250112 20:58:22.352860 17322 raft_consensus.cc:1270] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 LEARNER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:22.348260 16785 raft_consensus.cc:1270] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250112 20:58:22.356879 18185 consensus_queue.cc:1035] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:22.358085 18226 consensus_queue.cc:1035] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:22.359860 18239 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.365949 17324 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.366108 16632 raft_consensus.cc:2949] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.369995 18226 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.374997 17323 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.377262 16521 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet 5b3206c0399f4057928460c7a6f04912 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:22.385059 16537 catalog_manager.cc:5526] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb reported cstate change: config changed from index 4 to 5, VOTER c29b243acf2b4f248606563fc96efc7d (127.15.23.131) evicted. New cstate: current_term: 2 leader_uuid: "018d03f043b542b9b63491a77523968d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.392175 16785 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.400000 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3206c0399f4057928460c7a6f04912 with delete_type TABLET_DATA_TOMBSTONED (TS c29b243acf2b4f248606563fc96efc7d not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:55574
I20250112 20:58:22.401369 18234 tablet_replica.cc:331] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:22.402513 18234 raft_consensus.cc:2238] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:22.411741 18234 raft_consensus.cc:2267] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:22.421298 16711 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.425526 18234 ts_tablet_manager.cc:1905] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:22.427457 16532 catalog_manager.cc:5526] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 reported cstate change: config changed from index 2 to 3, 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } attrs { replace: true } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.444694 18234 ts_tablet_manager.cc:1918] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20250112 20:58:22.445171 18234 log.cc:1198] T 5b3206c0399f4057928460c7a6f04912 P c29b243acf2b4f248606563fc96efc7d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/wals/5b3206c0399f4057928460c7a6f04912
I20250112 20:58:22.446538 16524 catalog_manager.cc:4872] TS c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): tablet 5b3206c0399f4057928460c7a6f04912 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:22.447053 16632 consensus_queue.cc:237] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:22.452998 17323 raft_consensus.cc:1270] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:22.453253 16785 raft_consensus.cc:1270] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Refusing update from remote peer 80770068c3a34f918765f54dbe3a94eb: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20250112 20:58:22.455490 18161 consensus_queue.cc:1035] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:22.456681 18208 consensus_queue.cc:1035] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:58:22.462574 18226 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.463886 16785 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.471611 16522 catalog_manager.cc:5039] ChangeConfig:REMOVE_PEER RPC for tablet f021c2fa4d3143ff899283a33cb24538 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250112 20:58:22.469832 17323 raft_consensus.cc:2949] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.478806 16532 catalog_manager.cc:5526] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d reported cstate change: config changed from index 3 to 4, VOTER 018d03f043b542b9b63491a77523968d (127.15.23.130) evicted. New cstate: current_term: 1 leader_uuid: "80770068c3a34f918765f54dbe3a94eb" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } } }
I20250112 20:58:22.487722 16697 tablet_service.cc:1514] Processing DeleteTablet for tablet f021c2fa4d3143ff899283a33cb24538 with delete_type TABLET_DATA_TOMBSTONED (TS 018d03f043b542b9b63491a77523968d not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38952
I20250112 20:58:22.488662 18241 tablet_replica.cc:331] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:22.489568 18241 raft_consensus.cc:2238] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:22.490231 18241 raft_consensus.cc:2267] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:22.493597 18241 ts_tablet_manager.cc:1905] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250112 20:58:22.506220 18241 ts_tablet_manager.cc:1918] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20250112 20:58:22.506479 18241 log.cc:1198] T f021c2fa4d3143ff899283a33cb24538 P 018d03f043b542b9b63491a77523968d: Deleting WAL directory at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/wals/f021c2fa4d3143ff899283a33cb24538
I20250112 20:58:22.507436 16521 catalog_manager.cc:4872] TS 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): tablet f021c2fa4d3143ff899283a33cb24538 (table test-workload [id=7fe248d2bb09479ba0878b9c42545c37]) successfully deleted
I20250112 20:58:22.520761 16784 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250112 20:58:22.521914 18227 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20250112 20:58:22.539111 16636 raft_consensus.cc:1270] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Refusing update from remote peer 1b6f6e362d6c451d865ec54e6ce5dfe1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20250112 20:58:22.542188 18227 consensus_queue.cc:1035] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20250112 20:58:22.982885 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:23.020040 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:58:23.021819 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:23.022337 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:23.051226 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:23.083917 15454 tablet_server.cc:178] TabletServer@127.15.23.129:0 shutting down...
W20250112 20:58:23.091596 16678 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250112 20:58:23.096336 16678 consensus_peers.cc:487] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.113509 17288 consensus_peers.cc:487] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.130962 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:58:23.132259 15454 tablet_replica.cc:331] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.132937 15454 raft_consensus.cc:2238] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.133316 15454 raft_consensus.cc:2267] T 4799def22f754c9881fd8245eb8f75b3 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.134905 15454 tablet_replica.cc:331] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.135478 15454 raft_consensus.cc:2238] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.135895 15454 raft_consensus.cc:2267] T 7ef14661bd7346f68101fe2a0fb064bb P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.137739 15454 tablet_replica.cc:331] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.138253 15454 raft_consensus.cc:2238] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.138602 15454 raft_consensus.cc:2267] T 82159eb8aadb4973b65d7edc4f417d84 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.139237 16752 consensus_peers.cc:487] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.140089 15454 tablet_replica.cc:331] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.140671 15454 raft_consensus.cc:2238] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.141088 15454 raft_consensus.cc:2267] T e1773dc29c9e4d83a01c6b56f7b277d5 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.143195 15454 tablet_replica.cc:331] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.143893 15454 raft_consensus.cc:2238] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.144310 15454 raft_consensus.cc:2267] T aab6f0844afa43d8b7af8e3705d09c40 P 80770068c3a34f918765f54dbe3a94eb [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.146028 15454 tablet_replica.cc:331] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.146582 15454 raft_consensus.cc:2238] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.146967 15454 raft_consensus.cc:2267] T abeaacbf8c794ec89adddeb5f906f096 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.148521 15454 tablet_replica.cc:331] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.149142 15454 raft_consensus.cc:2238] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.149963 15454 raft_consensus.cc:2267] T f021c2fa4d3143ff899283a33cb24538 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.153363 15454 tablet_replica.cc:331] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.153846 15454 raft_consensus.cc:2238] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.154343 15454 raft_consensus.cc:2267] T b1d0b3432b644821ba556f59a893d40a P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.155877 15454 tablet_replica.cc:331] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.156342 15454 raft_consensus.cc:2238] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.156811 15454 raft_consensus.cc:2267] T 3d2477026d8d4fac945714c894cca485 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.158689 15454 tablet_replica.cc:331] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.159325 15454 raft_consensus.cc:2238] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.159876 15454 raft_consensus.cc:2267] T e23f9778f8494e11b9d942035720e363 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.161759 16752 consensus_peers.cc:487] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.162153 15454 tablet_replica.cc:331] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.163360 15454 raft_consensus.cc:2238] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.164165 15454 raft_consensus.cc:2267] T 81567654a26f435795e38a761d26bfa7 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.164472 16752 consensus_peers.cc:487] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.165707 15454 tablet_replica.cc:331] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.166328 15454 raft_consensus.cc:2238] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.166770 15454 raft_consensus.cc:2267] T 0b0ecdb5a6684883976581f48f328035 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.168795 15454 tablet_replica.cc:331] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.169404 15454 raft_consensus.cc:2238] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.170130 15454 raft_consensus.cc:2267] T 5ffbc5c73802406893d78f89cf5c145c P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.171604 15454 tablet_replica.cc:331] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.172089 15454 raft_consensus.cc:2238] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.172747 15454 raft_consensus.cc:2267] T 8e161101273b4b61a7235af454d77ca1 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.174301 15454 tablet_replica.cc:331] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.174782 15454 raft_consensus.cc:2238] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.175138 15454 raft_consensus.cc:2267] T 8c10f3f7e0f74c9f988fa01e7843e961 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.176597 15454 tablet_replica.cc:331] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.177079 15454 raft_consensus.cc:2238] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.177404 15454 raft_consensus.cc:2267] T df9c04a57cac449e8efae9c7689ad690 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.178762 15454 tablet_replica.cc:331] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.179219 15454 raft_consensus.cc:2238] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.179816 15454 raft_consensus.cc:2267] T 1b9eab84e9c44a4aae8715e7735721d8 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.181386 15454 tablet_replica.cc:331] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.181854 15454 raft_consensus.cc:2238] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.182240 15454 raft_consensus.cc:2267] T c1a1a7f345014fa7aff0f0cc0160789f P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.183687 15454 tablet_replica.cc:331] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.184162 15454 raft_consensus.cc:2238] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.184885 15454 raft_consensus.cc:2267] T 3f7ad2e8f5064918bd15be2a608c1db4 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.186400 15454 tablet_replica.cc:331] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.186972 15454 raft_consensus.cc:2238] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.187616 15454 raft_consensus.cc:2267] T a5ffa8e639d640a3b7f19d9857148f40 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.189738 15454 tablet_replica.cc:331] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.190379 15454 raft_consensus.cc:2238] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.191102 15454 raft_consensus.cc:2267] T e19f4890a44f4f088549ad693bafbf8d P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.192925 15454 tablet_replica.cc:331] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.193374 15454 raft_consensus.cc:2238] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.193930 15454 raft_consensus.cc:2267] T 963bd285f345421581941f10dbe8b90c P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.195410 15454 tablet_replica.cc:331] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.195839 15454 raft_consensus.cc:2238] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.196244 15454 raft_consensus.cc:2267] T 0e510cf639924856a759b20574cb60be P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.198041 15454 tablet_replica.cc:331] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.198522 15454 raft_consensus.cc:2238] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.198915 15454 raft_consensus.cc:2267] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.200400 15454 tablet_replica.cc:331] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.200991 15454 raft_consensus.cc:2238] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.201624 15454 raft_consensus.cc:2267] T 1b1cfa7b34e542098452925ed942c2bb P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.203047 15454 tablet_replica.cc:331] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.203480 15454 raft_consensus.cc:2238] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.203790 15454 raft_consensus.cc:2267] T 807445e11d91471bbfa7a55fcb99f774 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.205494 15454 tablet_replica.cc:331] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.205994 15454 raft_consensus.cc:2238] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.206303 15454 raft_consensus.cc:2267] T 7a0f7e20973149ed977de7cdd902fc39 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.207600 15454 tablet_replica.cc:331] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.208027 15454 raft_consensus.cc:2238] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.208338 15454 raft_consensus.cc:2267] T 7d37ea77e5934355b580ed367659c16c P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.209780 15454 tablet_replica.cc:331] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.210197 15454 raft_consensus.cc:2238] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.210808 15454 raft_consensus.cc:2267] T d979f5162b594f329ba9439c121741de P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.212252 15454 tablet_replica.cc:331] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.212729 15454 raft_consensus.cc:2238] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.213176 15454 raft_consensus.cc:2267] T 142fbcc35085432abdb80a4fad031c67 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.215076 15454 tablet_replica.cc:331] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.215505 15454 raft_consensus.cc:2238] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.215835 15454 raft_consensus.cc:2267] T abd904e2af24417f8b366444ad932e30 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.217360 15454 tablet_replica.cc:331] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.217811 15454 raft_consensus.cc:2238] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.218170 15454 raft_consensus.cc:2267] T 90bf3756af284af8b3ec4d29bbf5b972 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.219710 15454 tablet_replica.cc:331] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
W20250112 20:58:23.220103 17288 consensus_peers.cc:487] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.220137 15454 raft_consensus.cc:2238] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.221231 15454 raft_consensus.cc:2267] T 680c875f5d264744aef97e8dd9c304b9 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.223246 15454 tablet_replica.cc:331] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
W20250112 20:58:23.223526 16752 consensus_peers.cc:487] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.223924 15454 raft_consensus.cc:2238] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.224500 15454 raft_consensus.cc:2267] T 9a21c93f28fc4ab3be409b95204b86dc P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.225999 15454 tablet_replica.cc:331] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.226413 15454 raft_consensus.cc:2238] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.226774 15454 raft_consensus.cc:2267] T 5b3206c0399f4057928460c7a6f04912 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.228652 15454 tablet_replica.cc:331] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.229075 15454 raft_consensus.cc:2238] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.229380 15454 raft_consensus.cc:2267] T 3c5b79af971640d891e05584ff40dd7b P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.230777 15454 tablet_replica.cc:331] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.231169 15454 raft_consensus.cc:2238] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.231599 15454 raft_consensus.cc:2267] T cc2820c3d5974dfcb2230e18411e6493 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.232954 15454 tablet_replica.cc:331] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.233388 15454 raft_consensus.cc:2238] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.233757 15454 raft_consensus.cc:2267] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.235193 15454 tablet_replica.cc:331] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.235630 15454 raft_consensus.cc:2238] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.235975 15454 raft_consensus.cc:2267] T f3b1a80e42504299a3970ea4437b8a13 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.237305 15454 tablet_replica.cc:331] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.237696 15454 raft_consensus.cc:2238] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.238005 15454 raft_consensus.cc:2267] T 10e3fd35bff34f4bbaf6763ca8651656 P 80770068c3a34f918765f54dbe3a94eb [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.239383 15454 tablet_replica.cc:331] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.239930 15454 raft_consensus.cc:2238] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.240760 15454 raft_consensus.cc:2267] T d8a2acedfaa2476596c2ec8bb689dcce P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.242702 16752 consensus_peers.cc:487] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.242960 15454 tablet_replica.cc:331] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
W20250112 20:58:23.243546 17288 consensus_peers.cc:487] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.243618 15454 raft_consensus.cc:2238] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.244163 15454 raft_consensus.cc:2267] T 153749bb76c146f2ba4e7d07efbc1497 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.245869 15454 tablet_replica.cc:331] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.246277 15454 raft_consensus.cc:2238] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.246876 15454 raft_consensus.cc:2267] T 132239c0c07a4c2f9334e84188f329b3 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.248312 15454 tablet_replica.cc:331] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.248749 15454 raft_consensus.cc:2238] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.249109 15454 raft_consensus.cc:2267] T 1a3f67e9a10d448f93128a7f95344801 P 80770068c3a34f918765f54dbe3a94eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.250591 15454 tablet_replica.cc:331] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb: stopping tablet replica
I20250112 20:58:23.250974 15454 raft_consensus.cc:2238] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.251619 15454 raft_consensus.cc:2267] T 03cbd278697d4f6780211b21028f6d18 P 80770068c3a34f918765f54dbe3a94eb [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.291997 16752 consensus_peers.cc:487] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.293135 16678 consensus_peers.cc:487] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.325045 15454 tablet_server.cc:195] TabletServer@127.15.23.129:0 shutdown complete.
W20250112 20:58:23.331107 16752 consensus_peers.cc:487] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.360445 17288 consensus_peers.cc:487] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.370657 17288 consensus_peers.cc:487] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.379258 16678 consensus_peers.cc:487] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.380043 17288 consensus_peers.cc:487] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.409375 16678 consensus_peers.cc:487] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.419705 15454 tablet_server.cc:178] TabletServer@127.15.23.130:0 shutting down...
W20250112 20:58:23.424971 16678 consensus_peers.cc:487] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.434307 17288 consensus_peers.cc:487] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.438239 17287 consensus_peers.cc:487] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.455883 16678 consensus_peers.cc:487] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.464319 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:58:23.465229 15454 tablet_replica.cc:331] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.466113 15454 raft_consensus.cc:2238] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.466718 15454 raft_consensus.cc:2267] T 4799def22f754c9881fd8245eb8f75b3 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.468737 15454 tablet_replica.cc:331] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.469288 15454 raft_consensus.cc:2238] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.469794 15454 raft_consensus.cc:2267] T 1f61fd1d8bf44f64b73ade11c0224df0 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.471974 15454 tablet_replica.cc:331] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.472847 15454 raft_consensus.cc:2238] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.473616 15454 raft_consensus.cc:2267] T abeaacbf8c794ec89adddeb5f906f096 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.475400 15454 tablet_replica.cc:331] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.476064 15454 raft_consensus.cc:2238] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.476902 15454 raft_consensus.cc:2267] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.479125 15454 tablet_replica.cc:331] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.479861 15454 raft_consensus.cc:2238] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.480741 15454 raft_consensus.cc:2267] T cc2820c3d5974dfcb2230e18411e6493 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.482805 15454 tablet_replica.cc:331] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
W20250112 20:58:23.483356 16752 consensus_peers.cc:487] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.483516 15454 raft_consensus.cc:2238] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.485255 15454 raft_consensus.cc:2267] T 0b0ecdb5a6684883976581f48f328035 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.487212 15454 tablet_replica.cc:331] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.487713 15454 raft_consensus.cc:2238] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.488067 15454 raft_consensus.cc:2267] T 81567654a26f435795e38a761d26bfa7 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.489647 15454 tablet_replica.cc:331] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.490188 15454 raft_consensus.cc:2238] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.491111 15454 raft_consensus.cc:2267] T e23f9778f8494e11b9d942035720e363 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.493206 16678 consensus_peers.cc:487] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.493505 15454 tablet_replica.cc:331] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.494189 15454 raft_consensus.cc:2238] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.495065 15454 raft_consensus.cc:2267] T df9c04a57cac449e8efae9c7689ad690 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.495609 16678 consensus_peers.cc:487] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.497316 15454 tablet_replica.cc:331] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d: stopping tablet replica
W20250112 20:58:23.498000 17288 consensus_peers.cc:487] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.498127 15454 raft_consensus.cc:2238] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.498950 15454 raft_consensus.cc:2267] T c1a1a7f345014fa7aff0f0cc0160789f P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.500473 15454 tablet_replica.cc:331] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.500964 15454 raft_consensus.cc:2238] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.501417 15454 raft_consensus.cc:2267] T 7a13e4117fd64b15a4cd60943e638999 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.503345 15454 tablet_replica.cc:331] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
W20250112 20:58:23.503515 17287 consensus_peers.cc:487] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.503968 15454 raft_consensus.cc:2238] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.504468 15454 raft_consensus.cc:2267] T e1773dc29c9e4d83a01c6b56f7b277d5 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.506448 15454 tablet_replica.cc:331] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.507146 15454 raft_consensus.cc:2238] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 4 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.507951 15454 raft_consensus.cc:2267] T aab6f0844afa43d8b7af8e3705d09c40 P 018d03f043b542b9b63491a77523968d [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.509699 15454 tablet_replica.cc:331] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.510180 15454 raft_consensus.cc:2238] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.510805 15454 raft_consensus.cc:2267] T 6c34b0c62d424c9e9f764d74f6baf1ba P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.512411 15454 tablet_replica.cc:331] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.512888 15454 raft_consensus.cc:2238] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.513352 15454 raft_consensus.cc:2267] T 3f7ad2e8f5064918bd15be2a608c1db4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.514871 15454 tablet_replica.cc:331] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.515349 15454 raft_consensus.cc:2238] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.515676 15454 raft_consensus.cc:2267] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.541994 15454 tablet_replica.cc:331] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
W20250112 20:58:23.656975 17287 consensus_peers.cc:487] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.660992 15454 raft_consensus.cc:2238] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.663075 15454 raft_consensus.cc:2267] T a5ffa8e639d640a3b7f19d9857148f40 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.667265 15454 tablet_replica.cc:331] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.668331 15454 raft_consensus.cc:2238] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.670450 15454 raft_consensus.cc:2267] T 963bd285f345421581941f10dbe8b90c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.670562 17287 consensus_peers.cc:487] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.671463 17287 consensus_peers.cc:487] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.674443 15454 tablet_replica.cc:331] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.676337 15454 raft_consensus.cc:2238] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.679158 15454 raft_consensus.cc:2267] T 305481b596b746b1a03df8d76e4057fb P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.684788 15454 tablet_replica.cc:331] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.686264 15454 raft_consensus.cc:2238] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.687436 15454 raft_consensus.cc:2267] T 7d37ea77e5934355b580ed367659c16c P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.690850 15454 tablet_replica.cc:331] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.692382 15454 raft_consensus.cc:2238] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.693296 15454 raft_consensus.cc:2267] T 04890c40fe454c4a91d3d12e4104200a P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.698518 15454 tablet_replica.cc:331] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.699841 15454 raft_consensus.cc:2238] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.701560 15454 raft_consensus.cc:2267] T d979f5162b594f329ba9439c121741de P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.704857 15454 tablet_replica.cc:331] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.705930 15454 raft_consensus.cc:2238] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.708061 15454 raft_consensus.cc:2267] T 44e603d3a09141d296df5d386a3b4d11 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.712204 15454 tablet_replica.cc:331] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.713279 15454 raft_consensus.cc:2238] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:23.713402 17287 consensus_peers.cc:487] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.714107 15454 raft_consensus.cc:2267] T 1b1cfa7b34e542098452925ed942c2bb P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.719461 15454 tablet_replica.cc:331] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
W20250112 20:58:23.716264 16752 consensus_peers.cc:487] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.721167 16752 consensus_peers.cc:487] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.721726 16752 consensus_peers.cc:487] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d -> Peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Couldn't send request to peer 80770068c3a34f918765f54dbe3a94eb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.720364 15454 raft_consensus.cc:2238] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.724094 15454 raft_consensus.cc:2267] T 7a0f7e20973149ed977de7cdd902fc39 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.728230 17287 consensus_peers.cc:487] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.731935 15454 tablet_replica.cc:331] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.733054 15454 raft_consensus.cc:2238] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.733803 15454 raft_consensus.cc:2267] T 43f63edc7cd344fc88b39d2a6b2fbceb P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.736968 15454 tablet_replica.cc:331] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.737879 15454 raft_consensus.cc:2238] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.738724 15454 raft_consensus.cc:2267] T 03cbd278697d4f6780211b21028f6d18 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.738840 16751 consensus_peers.cc:487] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.739701 16751 consensus_peers.cc:487] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.740417 16751 consensus_peers.cc:487] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.741127 16751 consensus_peers.cc:487] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.741875 15454 tablet_replica.cc:331] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.743234 15454 raft_consensus.cc:2238] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.744728 15454 raft_consensus.cc:2267] T b0f35368d307429d8a87ba0c17bb4caf P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.748468 15454 tablet_replica.cc:331] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.750916 15454 raft_consensus.cc:2238] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.751653 15454 raft_consensus.cc:2267] T 14c4d77e5fce4c0e92935eb3a168cb77 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.754792 15454 tablet_replica.cc:331] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d: stopping tablet replica
W20250112 20:58:23.754968 16751 consensus_peers.cc:487] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.755779 15454 raft_consensus.cc:2238] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.757465 15454 raft_consensus.cc:2267] T d8a2acedfaa2476596c2ec8bb689dcce P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.760934 15454 tablet_replica.cc:331] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.761809 15454 raft_consensus.cc:2238] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.762614 15454 raft_consensus.cc:2267] T 9a21c93f28fc4ab3be409b95204b86dc P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.765550 15454 tablet_replica.cc:331] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.766696 15454 raft_consensus.cc:2238] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.768132 15454 raft_consensus.cc:2267] T 5b3206c0399f4057928460c7a6f04912 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.771652 15454 tablet_replica.cc:331] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.772539 15454 raft_consensus.cc:2238] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.773308 15454 raft_consensus.cc:2267] T dd7c3a6eea9043f4b438c4e28aca966e P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.776201 15454 tablet_replica.cc:331] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.776978 15454 raft_consensus.cc:2238] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.777981 15454 raft_consensus.cc:2267] T 132239c0c07a4c2f9334e84188f329b3 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.780412 15454 tablet_replica.cc:331] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.781415 15454 raft_consensus.cc:2238] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.782240 15454 raft_consensus.cc:2267] T 1a3f67e9a10d448f93128a7f95344801 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.784989 15454 tablet_replica.cc:331] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.785677 15454 raft_consensus.cc:2238] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.786854 15454 raft_consensus.cc:2267] T 3c5b79af971640d891e05584ff40dd7b P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.789441 15454 tablet_replica.cc:331] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.790189 15454 raft_consensus.cc:2238] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.790957 15454 raft_consensus.cc:2267] T f3b1a80e42504299a3970ea4437b8a13 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.794462 15454 tablet_replica.cc:331] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.795236 15454 raft_consensus.cc:2238] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.797031 15454 raft_consensus.cc:2267] T 10e3fd35bff34f4bbaf6763ca8651656 P 018d03f043b542b9b63491a77523968d [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.799561 15454 tablet_replica.cc:331] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.800199 15454 raft_consensus.cc:2238] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.800853 15454 raft_consensus.cc:2267] T 142fbcc35085432abdb80a4fad031c67 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.803730 15454 tablet_replica.cc:331] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.804476 15454 raft_consensus.cc:2238] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.806344 15454 raft_consensus.cc:2267] T 153749bb76c146f2ba4e7d07efbc1497 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.809695 15454 tablet_replica.cc:331] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.810724 15454 raft_consensus.cc:2238] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.812240 15454 raft_consensus.cc:2267] T 86b544e7e4fa4e419949c833428c406b P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.815080 15454 tablet_replica.cc:331] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.815872 15454 raft_consensus.cc:2238] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:23.817467 15454 raft_consensus.cc:2267] T 1696e724d24e4ebbb185476ec5ff8c09 P 018d03f043b542b9b63491a77523968d [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.819657 17287 consensus_peers.cc:487] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.820158 15454 tablet_replica.cc:331] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.821019 15454 raft_consensus.cc:2238] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.822244 15454 raft_consensus.cc:2267] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:23.824856 15454 tablet_replica.cc:331] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d: stopping tablet replica
I20250112 20:58:23.825572 15454 raft_consensus.cc:2238] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:23.826283 15454 raft_consensus.cc:2267] T 90bf3756af284af8b3ec4d29bbf5b972 P 018d03f043b542b9b63491a77523968d [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:23.834197 17287 consensus_peers.cc:487] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.835069 16751 consensus_peers.cc:487] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.846222 16751 consensus_peers.cc:487] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.917060 17287 connection.cc:537] client connection to 127.15.23.130:38087 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250112 20:58:23.917484 17287 consensus_peers.cc:487] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:23.934048 15454 tablet_server.cc:195] TabletServer@127.15.23.130:0 shutdown complete.
W20250112 20:58:23.992219 17287 consensus_peers.cc:487] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:23.992866 16751 consensus_peers.cc:487] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:24.012248 16751 consensus_peers.cc:487] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d -> Peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Couldn't send request to peer 018d03f043b542b9b63491a77523968d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:24.044232 15454 tablet_server.cc:178] TabletServer@127.15.23.131:0 shutting down...
W20250112 20:58:24.052625 17290 consensus_peers.cc:487] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:24.085810 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:58:24.086786 15454 tablet_replica.cc:331] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.087868 15454 raft_consensus.cc:2238] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.088835 15454 raft_consensus.cc:2267] T 82159eb8aadb4973b65d7edc4f417d84 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.090919 15454 tablet_replica.cc:331] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.091786 15454 raft_consensus.cc:2238] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.092825 15454 raft_consensus.cc:2267] T 1f61fd1d8bf44f64b73ade11c0224df0 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.095502 15454 tablet_replica.cc:331] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.096220 15454 raft_consensus.cc:2238] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.097172 15454 raft_consensus.cc:2267] T 2bdb0f016b8946c582e26ab6e02cb9b1 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.099454 15454 tablet_replica.cc:331] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.100142 15454 raft_consensus.cc:2238] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.100525 15454 raft_consensus.cc:2267] T abeaacbf8c794ec89adddeb5f906f096 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.102138 15454 tablet_replica.cc:331] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.102800 15454 raft_consensus.cc:2238] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.103693 15454 raft_consensus.cc:2267] T 7ef14661bd7346f68101fe2a0fb064bb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.105989 15454 tablet_replica.cc:331] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.106755 15454 raft_consensus.cc:2238] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.107326 15454 raft_consensus.cc:2267] T f021c2fa4d3143ff899283a33cb24538 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.108856 15454 tablet_replica.cc:331] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.109290 15454 raft_consensus.cc:2238] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.109623 15454 raft_consensus.cc:2267] T 81567654a26f435795e38a761d26bfa7 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.111757 15454 tablet_replica.cc:331] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.112210 15454 raft_consensus.cc:2238] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.112573 15454 raft_consensus.cc:2267] T df9c04a57cac449e8efae9c7689ad690 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.114166 15454 tablet_replica.cc:331] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.114637 15454 raft_consensus.cc:2238] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.115051 15454 raft_consensus.cc:2267] T 0b0ecdb5a6684883976581f48f328035 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.116526 15454 tablet_replica.cc:331] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.116906 18227 raft_consensus.cc:491] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.117097 15454 raft_consensus.cc:2238] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.117969 15454 raft_consensus.cc:2267] T 8e161101273b4b61a7235af454d77ca1 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.117368 18227 raft_consensus.cc:513] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.119566 18227 leader_election.cc:290] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:24.120064 15454 tablet_replica.cc:331] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.120852 15454 raft_consensus.cc:2238] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.121358 15454 raft_consensus.cc:2267] T 8c10f3f7e0f74c9f988fa01e7843e961 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:24.123080 17290 leader_election.cc:336] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250112 20:58:24.123920 15454 tablet_replica.cc:331] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
W20250112 20:58:24.123986 17288 leader_election.cc:336] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:24.124372 17288 leader_election.cc:304] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:24.124522 15454 raft_consensus.cc:2238] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.124953 18227 raft_consensus.cc:2743] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250112 20:58:24.125536 15454 raft_consensus.cc:2267] T 3d2477026d8d4fac945714c894cca485 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.127245 15454 tablet_replica.cc:331] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.127691 15454 raft_consensus.cc:2238] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.129004 15454 raft_consensus.cc:2267] T b1d0b3432b644821ba556f59a893d40a P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.130986 15454 tablet_replica.cc:331] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.131462 15454 raft_consensus.cc:2238] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.131975 15454 raft_consensus.cc:2267] T 5ffbc5c73802406893d78f89cf5c145c P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.133867 15454 tablet_replica.cc:331] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.134635 15454 raft_consensus.cc:2238] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.135272 15454 raft_consensus.cc:2267] T aab6f0844afa43d8b7af8e3705d09c40 P c29b243acf2b4f248606563fc96efc7d [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.137502 15454 tablet_replica.cc:331] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.138020 15454 raft_consensus.cc:2238] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.138639 15454 raft_consensus.cc:2267] T 6c34b0c62d424c9e9f764d74f6baf1ba P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.140664 15454 tablet_replica.cc:331] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.141170 15454 raft_consensus.cc:2238] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.141793 15454 raft_consensus.cc:2267] T 1b9eab84e9c44a4aae8715e7735721d8 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.144021 15454 tablet_replica.cc:331] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.144582 15454 raft_consensus.cc:2238] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.145196 15454 raft_consensus.cc:2267] T 7a13e4117fd64b15a4cd60943e638999 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.146719 15454 tablet_replica.cc:331] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.147176 15454 raft_consensus.cc:2238] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.147730 15454 raft_consensus.cc:2267] T 1696e724d24e4ebbb185476ec5ff8c09 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.149279 15454 tablet_replica.cc:331] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.149758 15454 raft_consensus.cc:2238] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.150262 15454 raft_consensus.cc:2267] T abd904e2af24417f8b366444ad932e30 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.151782 15454 tablet_replica.cc:331] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.152266 15454 raft_consensus.cc:2238] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.152987 15454 raft_consensus.cc:2267] T 3e3b5e20d22c4251b1e2b91ca0479e2f P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.154531 15454 tablet_replica.cc:331] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.154969 15454 raft_consensus.cc:2238] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.155344 15454 raft_consensus.cc:2267] T 1b1cfa7b34e542098452925ed942c2bb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.156533 18227 raft_consensus.cc:491] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.157009 18227 raft_consensus.cc:513] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.157330 15454 tablet_replica.cc:331] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.157848 15454 raft_consensus.cc:2238] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.158267 15454 raft_consensus.cc:2267] T 807445e11d91471bbfa7a55fcb99f774 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.158501 18227 leader_election.cc:290] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:24.160827 15454 tablet_replica.cc:331] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
W20250112 20:58:24.161079 17290 leader_election.cc:336] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250112 20:58:24.161505 15454 raft_consensus.cc:2238] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Raft consensus shutting down.
W20250112 20:58:24.162055 17288 leader_election.cc:336] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:24.162395 15454 raft_consensus.cc:2267] T 7a0f7e20973149ed977de7cdd902fc39 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.162374 17288 leader_election.cc:304] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:24.163035 18227 raft_consensus.cc:2743] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.164949 15454 tablet_replica.cc:331] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.165552 15454 raft_consensus.cc:2238] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.165975 15454 raft_consensus.cc:2267] T 43f63edc7cd344fc88b39d2a6b2fbceb P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.168047 15454 tablet_replica.cc:331] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.168742 15454 raft_consensus.cc:2238] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.169776 15454 raft_consensus.cc:2267] T 305481b596b746b1a03df8d76e4057fb P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.171872 15454 tablet_replica.cc:331] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.172320 15454 raft_consensus.cc:2238] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.172998 15454 raft_consensus.cc:2267] T 7d37ea77e5934355b580ed367659c16c P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.174841 15454 tablet_replica.cc:331] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.175539 15454 raft_consensus.cc:2238] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.176530 15454 raft_consensus.cc:2267] T 04890c40fe454c4a91d3d12e4104200a P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.178678 15454 tablet_replica.cc:331] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.179311 15454 raft_consensus.cc:2238] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.179823 15454 raft_consensus.cc:2267] T 44e603d3a09141d296df5d386a3b4d11 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.181859 15454 tablet_replica.cc:331] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.182422 15454 raft_consensus.cc:2238] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.182859 15454 raft_consensus.cc:2267] T 03cbd278697d4f6780211b21028f6d18 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:24.183862 17290 consensus_peers.cc:487] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:24.184873 15454 tablet_replica.cc:331] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.185385 15454 raft_consensus.cc:2238] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.186163 15454 raft_consensus.cc:2267] T b0f35368d307429d8a87ba0c17bb4caf P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.188228 15454 tablet_replica.cc:331] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.188751 15454 raft_consensus.cc:2238] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.189110 15454 raft_consensus.cc:2267] T 14c4d77e5fce4c0e92935eb3a168cb77 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.190622 15454 tablet_replica.cc:331] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.191154 15454 raft_consensus.cc:2238] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.191872 15454 raft_consensus.cc:2267] T 86b544e7e4fa4e419949c833428c406b P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.193372 15454 tablet_replica.cc:331] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.193890 15454 raft_consensus.cc:2238] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.194489 15454 raft_consensus.cc:2267] T e19f4890a44f4f088549ad693bafbf8d P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.195971 15454 tablet_replica.cc:331] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.196378 15454 raft_consensus.cc:2238] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.197072 15454 raft_consensus.cc:2267] T 0e510cf639924856a759b20574cb60be P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.198675 15454 tablet_replica.cc:331] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.199151 15454 raft_consensus.cc:2238] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.199806 15454 raft_consensus.cc:2267] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.201606 15454 tablet_replica.cc:331] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.202061 15454 raft_consensus.cc:2238] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.202747 15454 raft_consensus.cc:2267] T f3b1a80e42504299a3970ea4437b8a13 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.204336 15454 tablet_replica.cc:331] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.204813 15454 raft_consensus.cc:2238] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.205220 15454 raft_consensus.cc:2267] T 10e3fd35bff34f4bbaf6763ca8651656 P c29b243acf2b4f248606563fc96efc7d [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.206869 15454 tablet_replica.cc:331] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.207320 15454 raft_consensus.cc:2238] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.207935 15454 raft_consensus.cc:2267] T 90bf3756af284af8b3ec4d29bbf5b972 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.209661 15454 tablet_replica.cc:331] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.210072 15454 raft_consensus.cc:2238] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.210433 15454 raft_consensus.cc:2267] T 3c5b79af971640d891e05584ff40dd7b P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.211848 15454 tablet_replica.cc:331] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.212291 15454 raft_consensus.cc:2238] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.212724 15454 raft_consensus.cc:2267] T 153749bb76c146f2ba4e7d07efbc1497 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.214244 15454 tablet_replica.cc:331] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.214668 15454 raft_consensus.cc:2238] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:24.215308 15454 raft_consensus.cc:2267] T 1a3f67e9a10d448f93128a7f95344801 P c29b243acf2b4f248606563fc96efc7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.216920 15454 tablet_replica.cc:331] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.217337 15454 raft_consensus.cc:2238] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.217865 15454 raft_consensus.cc:2267] T dd7c3a6eea9043f4b438c4e28aca966e P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.218894 18265 raft_consensus.cc:491] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.219259 18265 raft_consensus.cc:513] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.219579 15454 tablet_replica.cc:331] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d: stopping tablet replica
I20250112 20:58:24.220106 15454 raft_consensus.cc:2238] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:24.220872 15454 raft_consensus.cc:2267] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:24.221259 18265 leader_election.cc:290] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 1b6f6e362d6c451d865ec54e6ce5dfe1 (127.15.23.132:43633)
I20250112 20:58:24.222601 17322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "680c875f5d264744aef97e8dd9c304b9" candidate_uuid: "c29b243acf2b4f248606563fc96efc7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" is_pre_election: true
W20250112 20:58:24.224731 16752 leader_election.cc:336] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:24.225037 16752 leader_election.cc:304] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c29b243acf2b4f248606563fc96efc7d; no voters: 1b6f6e362d6c451d865ec54e6ce5dfe1, 80770068c3a34f918765f54dbe3a94eb
W20250112 20:58:24.225517 16752 raft_consensus.cc:2681] T 680c875f5d264744aef97e8dd9c304b9 P c29b243acf2b4f248606563fc96efc7d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20250112 20:58:24.274816 17290 consensus_peers.cc:487] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:24.304800 17290 consensus_peers.cc:487] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:24.317039 18229 raft_consensus.cc:491] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.317485 18229 raft_consensus.cc:513] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.318941 18229 leader_election.cc:290] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:58:24.321790 17288 leader_election.cc:336] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:24.322214 17287 leader_election.cc:336] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
I20250112 20:58:24.322505 17287 leader_election.cc:304] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.322971 18229 raft_consensus.cc:2743] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.376019 18227 raft_consensus.cc:491] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.376526 18227 raft_consensus.cc:513] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.378758 18227 leader_election.cc:290] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:58:24.382022 17288 leader_election.cc:336] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:24.383520 17287 leader_election.cc:336] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
I20250112 20:58:24.383879 17287 leader_election.cc:304] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.384483 18227 raft_consensus.cc:2743] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250112 20:58:24.416075 17290 consensus_peers.cc:487] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:24.454087 17290 consensus_peers.cc:487] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:24.472183 18227 raft_consensus.cc:491] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.472683 18227 raft_consensus.cc:513] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.474251 18227 leader_election.cc:290] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:24.476951 17288 leader_election.cc:336] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:24.477319 17290 leader_election.cc:336] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
I20250112 20:58:24.477595 17290 leader_election.cc:304] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:24.478086 18227 raft_consensus.cc:2743] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.485615 18227 raft_consensus.cc:491] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.485960 18227 raft_consensus.cc:513] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.487360 18227 leader_election.cc:290] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:24.489948 17288 leader_election.cc:336] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:24.490418 17290 leader_election.cc:336] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
I20250112 20:58:24.490715 17290 leader_election.cc:304] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:24.491230 18227 raft_consensus.cc:2743] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.494418 18227 raft_consensus.cc:491] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.494943 18227 raft_consensus.cc:513] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
W20250112 20:58:24.495461 17290 consensus_peers.cc:487] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:24.496611 18227 leader_election.cc:290] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:58:24.499693 17287 leader_election.cc:336] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
W20250112 20:58:24.501170 17288 leader_election.cc:336] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:24.501480 17288 leader_election.cc:304] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.501953 18227 raft_consensus.cc:2743] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.529557 18227 raft_consensus.cc:491] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 018d03f043b542b9b63491a77523968d)
I20250112 20:58:24.529940 18227 raft_consensus.cc:513] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.531522 18227 leader_election.cc:290] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:58:24.534467 17288 leader_election.cc:336] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:24.534916 17287 leader_election.cc:336] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
I20250112 20:58:24.535303 17287 leader_election.cc:304] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.535804 18227 raft_consensus.cc:2743] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250112 20:58:24.537998 17290 consensus_peers.cc:487] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:24.568415 17290 consensus_peers.cc:487] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 -> Peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Couldn't send request to peer c29b243acf2b4f248606563fc96efc7d. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:24.615355 18267 raft_consensus.cc:491] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.615877 18267 raft_consensus.cc:513] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.617465 18267 leader_election.cc:290] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:24.620944 18267 raft_consensus.cc:491] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 018d03f043b542b9b63491a77523968d)
W20250112 20:58:24.621050 17290 leader_election.cc:336] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
W20250112 20:58:24.621567 17288 leader_election.cc:336] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:24.621407 18267 raft_consensus.cc:513] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.621984 17288 leader_election.cc:304] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:24.622527 18227 raft_consensus.cc:491] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.623273 18267 leader_election.cc:290] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:24.623157 18227 raft_consensus.cc:513] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.622625 18229 raft_consensus.cc:2743] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.624186 18229 raft_consensus.cc:491] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.624644 18229 raft_consensus.cc:513] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.625147 18227 leader_election.cc:290] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:24.626449 18229 leader_election.cc:290] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:24.623323 18251 raft_consensus.cc:491] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.627103 18251 raft_consensus.cc:513] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
W20250112 20:58:24.627593 17287 leader_election.cc:336] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
W20250112 20:58:24.628067 17287 leader_election.cc:336] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
I20250112 20:58:24.628899 18251 leader_election.cc:290] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:24.630596 17287 leader_election.cc:336] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
W20250112 20:58:24.631516 17290 leader_election.cc:336] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
W20250112 20:58:24.631697 17288 leader_election.cc:336] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:24.631892 17290 leader_election.cc:304] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:24.632069 17288 leader_election.cc:304] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.632458 18251 raft_consensus.cc:2743] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.632681 18229 raft_consensus.cc:2743] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250112 20:58:24.632815 17288 leader_election.cc:336] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:24.633235 17288 leader_election.cc:304] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.633744 18229 raft_consensus.cc:2743] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250112 20:58:24.633816 17288 leader_election.cc:336] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:24.634469 17290 leader_election.cc:336] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
I20250112 20:58:24.634788 17290 leader_election.cc:304] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:24.635258 18229 raft_consensus.cc:2743] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250112 20:58:24.643074 18229 raft_consensus.cc:491] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 018d03f043b542b9b63491a77523968d)
I20250112 20:58:24.643419 18229 raft_consensus.cc:513] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.644822 18229 leader_election.cc:290] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:24.647444 17287 leader_election.cc:336] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
W20250112 20:58:24.647848 17290 leader_election.cc:336] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
I20250112 20:58:24.648149 17290 leader_election.cc:304] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:24.648643 18229 raft_consensus.cc:2743] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.688431 18229 raft_consensus.cc:491] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 80770068c3a34f918765f54dbe3a94eb)
I20250112 20:58:24.688895 18229 raft_consensus.cc:513] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.690410 18229 leader_election.cc:290] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:58:24.693223 17288 leader_election.cc:336] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:24.693560 17287 leader_election.cc:336] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
I20250112 20:58:24.693868 17287 leader_election.cc:304] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.694365 18229 raft_consensus.cc:2743] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250112 20:58:24.765977 16809 debug-util.cc:398] Leaking SignalData structure 0x7b080017a180 after lost signal to thread 15457
W20250112 20:58:24.766994 16809 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd880 after lost signal to thread 16571
W20250112 20:58:24.768992 16809 debug-util.cc:398] Leaking SignalData structure 0x7b08001104c0 after lost signal to thread 17349
I20250112 20:58:24.856559 18227 raft_consensus.cc:491] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 018d03f043b542b9b63491a77523968d)
I20250112 20:58:24.857235 18227 raft_consensus.cc:513] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.859521 18227 leader_election.cc:290] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:58:24.862820 17288 leader_election.cc:336] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:24.867704 17287 leader_election.cc:336] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
I20250112 20:58:24.868080 17287 leader_election.cc:304] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.868991 18229 raft_consensus.cc:491] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 018d03f043b542b9b63491a77523968d)
I20250112 20:58:24.869343 18227 raft_consensus.cc:2743] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:24.869519 18229 raft_consensus.cc:513] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:24.871676 18229 leader_election.cc:290] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
W20250112 20:58:24.874730 17287 leader_election.cc:336] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
W20250112 20:58:24.875134 17288 leader_election.cc:336] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:24.875543 17288 leader_election.cc:304] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:24.876173 18229 raft_consensus.cc:2743] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250112 20:58:25.196375 18227 raft_consensus.cc:491] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c29b243acf2b4f248606563fc96efc7d)
I20250112 20:58:25.196822 18227 raft_consensus.cc:513] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:25.198204 18227 leader_election.cc:290] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:25.199791 18227 raft_consensus.cc:491] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c29b243acf2b4f248606563fc96efc7d)
I20250112 20:58:25.200222 18227 raft_consensus.cc:513] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
W20250112 20:58:25.201469 17290 leader_election.cc:336] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
W20250112 20:58:25.201977 17287 leader_election.cc:336] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
I20250112 20:58:25.202358 17287 leader_election.cc:304] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:25.203368 18251 raft_consensus.cc:2743] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:25.204296 18227 leader_election.cc:290] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:25.206321 17288 leader_election.cc:336] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:25.206703 17290 leader_election.cc:336] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
I20250112 20:58:25.206975 17290 leader_election.cc:304] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:25.207406 18227 raft_consensus.cc:2743] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:25.219470 15454 tablet_server.cc:195] TabletServer@127.15.23.131:0 shutdown complete.
I20250112 20:58:25.242343 18227 raft_consensus.cc:491] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 018d03f043b542b9b63491a77523968d)
I20250112 20:58:25.242861 18227 raft_consensus.cc:513] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:25.244439 18227 leader_election.cc:290] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:25.247758 17287 leader_election.cc:336] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
W20250112 20:58:25.248517 17290 leader_election.cc:336] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
I20250112 20:58:25.248934 17290 leader_election.cc:304] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:25.249621 18227 raft_consensus.cc:2743] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250112 20:58:25.302244 15454 tablet_server.cc:178] TabletServer@127.15.23.132:0 shutting down...
I20250112 20:58:25.304656 18227 raft_consensus.cc:491] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 018d03f043b542b9b63491a77523968d)
I20250112 20:58:25.305219 18227 raft_consensus.cc:513] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:25.306504 18267 raft_consensus.cc:491] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 018d03f043b542b9b63491a77523968d)
I20250112 20:58:25.306968 18267 raft_consensus.cc:513] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:25.307291 18227 leader_election.cc:290] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), 018d03f043b542b9b63491a77523968d (127.15.23.130:38087)
I20250112 20:58:25.309007 18267 leader_election.cc:290] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:25.311738 17288 leader_election.cc:336] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
W20250112 20:58:25.313328 17287 leader_election.cc:336] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
I20250112 20:58:25.313724 17287 leader_election.cc:304] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, 80770068c3a34f918765f54dbe3a94eb
I20250112 20:58:25.314240 18267 raft_consensus.cc:2743] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250112 20:58:25.314392 17287 leader_election.cc:336] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
W20250112 20:58:25.314862 17290 leader_election.cc:336] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
I20250112 20:58:25.315240 17290 leader_election.cc:304] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:25.315806 18267 raft_consensus.cc:491] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c29b243acf2b4f248606563fc96efc7d)
I20250112 20:58:25.316785 18227 raft_consensus.cc:2743] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:25.316732 18267 raft_consensus.cc:513] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "018d03f043b542b9b63491a77523968d" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 38087 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:25.318691 18227 raft_consensus.cc:491] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c29b243acf2b4f248606563fc96efc7d)
I20250112 20:58:25.319123 18267 leader_election.cc:290] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 018d03f043b542b9b63491a77523968d (127.15.23.130:38087), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:25.319128 18227 raft_consensus.cc:513] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:25.321208 18227 leader_election.cc:290] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:25.346860 17290 leader_election.cc:336] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
W20250112 20:58:25.348551 17290 leader_election.cc:336] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
W20250112 20:58:25.348920 17287 leader_election.cc:336] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 018d03f043b542b9b63491a77523968d (127.15.23.130:38087): Network error: Client connection negotiation failed: client connection to 127.15.23.130:38087: connect: Connection refused (error 111)
W20250112 20:58:25.349102 17288 leader_election.cc:336] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:25.349401 18227 raft_consensus.cc:491] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c29b243acf2b4f248606563fc96efc7d)
I20250112 20:58:25.349710 17288 leader_election.cc:304] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:25.349843 18227 raft_consensus.cc:513] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:25.350579 18267 raft_consensus.cc:2743] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250112 20:58:25.351106 18251 raft_consensus.cc:491] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c29b243acf2b4f248606563fc96efc7d)
I20250112 20:58:25.351572 18227 leader_election.cc:290] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
I20250112 20:58:25.351500 18251 raft_consensus.cc:513] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "80770068c3a34f918765f54dbe3a94eb" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 41463 } } peers { permanent_uuid: "c29b243acf2b4f248606563fc96efc7d" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 38713 } } peers { permanent_uuid: "1b6f6e362d6c451d865ec54e6ce5dfe1" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 43633 } attrs { promote: false } }
I20250112 20:58:25.349363 17287 leader_election.cc:304] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 018d03f043b542b9b63491a77523968d, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:25.352648 18227 raft_consensus.cc:2743] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:25.353467 18251 leader_election.cc:290] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463), c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713)
W20250112 20:58:25.355121 17290 leader_election.cc:336] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
W20250112 20:58:25.358991 17288 leader_election.cc:336] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:25.360325 17288 leader_election.cc:304] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:25.361270 18227 raft_consensus.cc:2743] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250112 20:58:25.362072 17290 leader_election.cc:336] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c29b243acf2b4f248606563fc96efc7d (127.15.23.131:38713): Network error: Client connection negotiation failed: client connection to 127.15.23.131:38713: connect: Connection refused (error 111)
W20250112 20:58:25.364463 17288 leader_election.cc:336] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 80770068c3a34f918765f54dbe3a94eb (127.15.23.129:41463): Network error: Client connection negotiation failed: client connection to 127.15.23.129:41463: connect: Connection refused (error 111)
I20250112 20:58:25.366655 17288 leader_election.cc:304] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b6f6e362d6c451d865ec54e6ce5dfe1; no voters: 80770068c3a34f918765f54dbe3a94eb, c29b243acf2b4f248606563fc96efc7d
I20250112 20:58:25.367327 18227 raft_consensus.cc:2743] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250112 20:58:25.369474 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:58:25.370245 15454 tablet_replica.cc:331] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.371273 15454 raft_consensus.cc:2238] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.371915 15454 raft_consensus.cc:2267] T c1a1a7f345014fa7aff0f0cc0160789f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.373878 15454 tablet_replica.cc:331] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.374395 15454 raft_consensus.cc:2238] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.374811 15454 raft_consensus.cc:2267] T f021c2fa4d3143ff899283a33cb24538 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.376475 15454 tablet_replica.cc:331] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.376996 15454 raft_consensus.cc:2238] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.377445 15454 raft_consensus.cc:2267] T 86b544e7e4fa4e419949c833428c406b P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.379087 15454 tablet_replica.cc:331] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.379581 15454 raft_consensus.cc:2238] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.380054 15454 raft_consensus.cc:2267] T 2bdb0f016b8946c582e26ab6e02cb9b1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.381731 15454 tablet_replica.cc:331] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.382230 15454 raft_consensus.cc:2238] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.382709 15454 raft_consensus.cc:2267] T 3f7ad2e8f5064918bd15be2a608c1db4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.384384 15454 tablet_replica.cc:331] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.384912 15454 raft_consensus.cc:2238] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.385402 15454 raft_consensus.cc:2267] T 6c34b0c62d424c9e9f764d74f6baf1ba P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.387022 15454 tablet_replica.cc:331] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.387601 15454 raft_consensus.cc:2238] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.388010 15454 raft_consensus.cc:2267] T d8a2acedfaa2476596c2ec8bb689dcce P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.390358 15454 tablet_replica.cc:331] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.391099 15454 raft_consensus.cc:2238] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.391646 15454 raft_consensus.cc:2267] T e23f9778f8494e11b9d942035720e363 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.393234 15454 tablet_replica.cc:331] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.393673 15454 raft_consensus.cc:2238] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.394671 15454 raft_consensus.cc:2267] T 680c875f5d264744aef97e8dd9c304b9 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.397133 15454 tablet_replica.cc:331] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.397740 15454 raft_consensus.cc:2238] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.398195 15454 raft_consensus.cc:2267] T a5ffa8e639d640a3b7f19d9857148f40 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.399756 15454 tablet_replica.cc:331] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.400413 15454 raft_consensus.cc:2238] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.400918 15454 raft_consensus.cc:2267] T 1b9eab84e9c44a4aae8715e7735721d8 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.402442 15454 tablet_replica.cc:331] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.402909 15454 raft_consensus.cc:2238] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.403577 15454 raft_consensus.cc:2267] T 7a13e4117fd64b15a4cd60943e638999 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.405745 15454 tablet_replica.cc:331] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.406487 15454 raft_consensus.cc:2238] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.407306 15454 raft_consensus.cc:2267] T e1773dc29c9e4d83a01c6b56f7b277d5 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.408993 15454 tablet_replica.cc:331] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.409430 15454 raft_consensus.cc:2238] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.409842 15454 raft_consensus.cc:2267] T 8e161101273b4b61a7235af454d77ca1 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.411478 15454 tablet_replica.cc:331] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.411936 15454 raft_consensus.cc:2238] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.412331 15454 raft_consensus.cc:2267] T 1696e724d24e4ebbb185476ec5ff8c09 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.413931 15454 tablet_replica.cc:331] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.414374 15454 raft_consensus.cc:2238] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.414779 15454 raft_consensus.cc:2267] T 3e3b5e20d22c4251b1e2b91ca0479e2f P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.416200 15454 tablet_replica.cc:331] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.416693 15454 raft_consensus.cc:2238] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.417304 15454 raft_consensus.cc:2267] T abd904e2af24417f8b366444ad932e30 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.418843 15454 tablet_replica.cc:331] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.419302 15454 raft_consensus.cc:2238] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.420001 15454 raft_consensus.cc:2267] T 14c4d77e5fce4c0e92935eb3a168cb77 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.424638 15454 tablet_replica.cc:331] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.425127 15454 raft_consensus.cc:2238] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.425458 15454 raft_consensus.cc:2267] T 82159eb8aadb4973b65d7edc4f417d84 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.427047 15454 tablet_replica.cc:331] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.427500 15454 raft_consensus.cc:2238] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.428218 15454 raft_consensus.cc:2267] T b17fe6b5636e4cc9a2e96751cd7a14c4 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.430284 15454 tablet_replica.cc:331] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.430992 15454 raft_consensus.cc:2238] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.431807 15454 raft_consensus.cc:2267] T 142fbcc35085432abdb80a4fad031c67 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.433485 15454 tablet_replica.cc:331] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.433943 15454 raft_consensus.cc:2238] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.434355 15454 raft_consensus.cc:2267] T 1f61fd1d8bf44f64b73ade11c0224df0 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.436208 15454 tablet_replica.cc:331] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.436827 15454 raft_consensus.cc:2238] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.437687 15454 raft_consensus.cc:2267] T 04890c40fe454c4a91d3d12e4104200a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.439683 15454 tablet_replica.cc:331] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.440244 15454 raft_consensus.cc:2238] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.440740 15454 raft_consensus.cc:2267] T 9b3eb6b1d35d4c23bb0732c8c18c56ab P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.442513 15454 tablet_replica.cc:331] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.442988 15454 raft_consensus.cc:2238] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.443437 15454 raft_consensus.cc:2267] T cc2820c3d5974dfcb2230e18411e6493 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.444993 15454 tablet_replica.cc:331] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.445437 15454 raft_consensus.cc:2238] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.445854 15454 raft_consensus.cc:2267] T 305481b596b746b1a03df8d76e4057fb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.447360 15454 tablet_replica.cc:331] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.447849 15454 raft_consensus.cc:2238] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.448562 15454 raft_consensus.cc:2267] T 44e603d3a09141d296df5d386a3b4d11 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.450356 15454 tablet_replica.cc:331] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.450826 15454 raft_consensus.cc:2238] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.451501 15454 raft_consensus.cc:2267] T 807445e11d91471bbfa7a55fcb99f774 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.453220 15454 tablet_replica.cc:331] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.453692 15454 raft_consensus.cc:2238] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.454284 15454 raft_consensus.cc:2267] T 43f63edc7cd344fc88b39d2a6b2fbceb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.455845 15454 tablet_replica.cc:331] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.456290 15454 raft_consensus.cc:2238] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.456708 15454 raft_consensus.cc:2267] T d979f5162b594f329ba9439c121741de P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.458253 15454 tablet_replica.cc:331] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.458746 15454 raft_consensus.cc:2238] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.459427 15454 raft_consensus.cc:2267] T 4799def22f754c9881fd8245eb8f75b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.461067 15454 tablet_replica.cc:331] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.461552 15454 raft_consensus.cc:2238] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.461977 15454 raft_consensus.cc:2267] T 0e510cf639924856a759b20574cb60be P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.463567 15454 tablet_replica.cc:331] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.464025 15454 raft_consensus.cc:2238] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.464372 15454 raft_consensus.cc:2267] T 7ef14661bd7346f68101fe2a0fb064bb P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.466044 15454 tablet_replica.cc:331] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.466511 15454 raft_consensus.cc:2238] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.466929 15454 raft_consensus.cc:2267] T b1d0b3432b644821ba556f59a893d40a P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.468448 15454 tablet_replica.cc:331] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.468926 15454 raft_consensus.cc:2238] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.469578 15454 raft_consensus.cc:2267] T 8c10f3f7e0f74c9f988fa01e7843e961 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.471252 15454 tablet_replica.cc:331] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.471712 15454 raft_consensus.cc:2238] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.472234 15454 raft_consensus.cc:2267] T 5ffbc5c73802406893d78f89cf5c145c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.473816 15454 tablet_replica.cc:331] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.474311 15454 raft_consensus.cc:2238] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.474740 15454 raft_consensus.cc:2267] T b0f35368d307429d8a87ba0c17bb4caf P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.476816 15454 tablet_replica.cc:331] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.477289 15454 raft_consensus.cc:2238] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.477710 15454 raft_consensus.cc:2267] T 132239c0c07a4c2f9334e84188f329b3 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.479212 15454 tablet_replica.cc:331] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.479666 15454 raft_consensus.cc:2238] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.480072 15454 raft_consensus.cc:2267] T 3d2477026d8d4fac945714c894cca485 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.481628 15454 tablet_replica.cc:331] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.482075 15454 raft_consensus.cc:2238] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.482753 15454 raft_consensus.cc:2267] T 9a21c93f28fc4ab3be409b95204b86dc P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.484324 15454 tablet_replica.cc:331] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.484848 15454 raft_consensus.cc:2238] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.485286 15454 raft_consensus.cc:2267] T 5b3206c0399f4057928460c7a6f04912 P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.486792 15454 tablet_replica.cc:331] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.487190 15454 raft_consensus.cc:2238] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.487809 15454 raft_consensus.cc:2267] T dd7c3a6eea9043f4b438c4e28aca966e P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.489403 15454 tablet_replica.cc:331] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.489826 15454 raft_consensus.cc:2238] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.490319 15454 raft_consensus.cc:2267] T 963bd285f345421581941f10dbe8b90c P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.492028 15454 tablet_replica.cc:331] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1: stopping tablet replica
I20250112 20:58:25.492461 15454 raft_consensus.cc:2238] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:25.492899 15454 raft_consensus.cc:2267] T e19f4890a44f4f088549ad693bafbf8d P 1b6f6e362d6c451d865ec54e6ce5dfe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.551648 15454 tablet_server.cc:195] TabletServer@127.15.23.132:0 shutdown complete.
I20250112 20:58:25.606385 15454 master.cc:537] Master@127.15.23.190:44397 shutting down...
I20250112 20:58:25.630537 15454 raft_consensus.cc:2238] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:25.631093 15454 raft_consensus.cc:2267] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:25.631410 15454 tablet_replica.cc:331] T 00000000000000000000000000000000 P 1f263944a9a44fb1abb8091c03bc537a: stopping tablet replica
I20250112 20:58:25.685425 15454 master.cc:559] Master@127.15.23.190:44397 shutdown complete.
[ OK ] LeaderRebalancerTest.AddTserver (52114 ms)
[ RUN ] LeaderRebalancerTest.RestartTserver
I20250112 20:58:25.771054 15454 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.23.190:45335
I20250112 20:58:25.772127 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:58:25.778185 18269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:58:25.779206 18270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:25.779716 15454 server_base.cc:1034] running on GCE node
W20250112 20:58:25.780745 18272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:25.781581 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:58:25.781776 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:58:25.781924 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715505781908 us; error 0 us; skew 500 ppm
I20250112 20:58:25.782423 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:58:25.788313 15454 webserver.cc:458] Webserver started at http://127.15.23.190:42807/ using document root <none> and password file <none>
I20250112 20:58:25.788830 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:58:25.788995 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:58:25.789250 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:58:25.790501 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/master-0-root/instance:
uuid: "d64fb54f69204f7287968b68ba6d048a"
format_stamp: "Formatted at 2025-01-12 20:58:25 on dist-test-slave-m2nm"
I20250112 20:58:25.795192 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250112 20:58:25.798522 18277 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:25.799211 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250112 20:58:25.799497 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/master-0-root
uuid: "d64fb54f69204f7287968b68ba6d048a"
format_stamp: "Formatted at 2025-01-12 20:58:25 on dist-test-slave-m2nm"
I20250112 20:58:25.799770 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:58:25.828236 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:58:25.829484 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:58:25.865103 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.190:45335
I20250112 20:58:25.865235 18328 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.190:45335 every 8 connection(s)
I20250112 20:58:25.868886 18329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:25.878268 18329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a: Bootstrap starting.
I20250112 20:58:25.882858 18329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:25.886827 18329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a: No bootstrap required, opened a new log
I20250112 20:58:25.888998 18329 raft_consensus.cc:357] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fb54f69204f7287968b68ba6d048a" member_type: VOTER }
I20250112 20:58:25.889477 18329 raft_consensus.cc:383] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:25.889731 18329 raft_consensus.cc:738] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64fb54f69204f7287968b68ba6d048a, State: Initialized, Role: FOLLOWER
I20250112 20:58:25.890247 18329 consensus_queue.cc:260] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fb54f69204f7287968b68ba6d048a" member_type: VOTER }
I20250112 20:58:25.890663 18329 raft_consensus.cc:397] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 20:58:25.890878 18329 raft_consensus.cc:491] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 20:58:25.891145 18329 raft_consensus.cc:3054] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:25.895022 18329 raft_consensus.cc:513] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fb54f69204f7287968b68ba6d048a" member_type: VOTER }
I20250112 20:58:25.895464 18329 leader_election.cc:304] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d64fb54f69204f7287968b68ba6d048a; no voters:
I20250112 20:58:25.896492 18329 leader_election.cc:290] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 20:58:25.896903 18332 raft_consensus.cc:2798] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:25.898185 18332 raft_consensus.cc:695] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 1 LEADER]: Becoming Leader. State: Replica: d64fb54f69204f7287968b68ba6d048a, State: Running, Role: LEADER
I20250112 20:58:25.898924 18332 consensus_queue.cc:237] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fb54f69204f7287968b68ba6d048a" member_type: VOTER }
I20250112 20:58:25.899566 18329 sys_catalog.cc:564] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [sys.catalog]: configured and running, proceeding with master startup.
I20250112 20:58:25.903287 18333 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d64fb54f69204f7287968b68ba6d048a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fb54f69204f7287968b68ba6d048a" member_type: VOTER } }
I20250112 20:58:25.904106 18333 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [sys.catalog]: This master's current role is: LEADER
I20250112 20:58:25.903901 18334 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d64fb54f69204f7287968b68ba6d048a. Latest consensus state: current_term: 1 leader_uuid: "d64fb54f69204f7287968b68ba6d048a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fb54f69204f7287968b68ba6d048a" member_type: VOTER } }
I20250112 20:58:25.904816 18334 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [sys.catalog]: This master's current role is: LEADER
I20250112 20:58:25.907481 18339 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 20:58:25.915215 18339 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 20:58:25.916181 15454 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250112 20:58:25.923350 18339 catalog_manager.cc:1348] Generated new cluster ID: 3bad7e88f7e542e5abebfacc02b3517f
I20250112 20:58:25.923624 18339 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 20:58:25.946934 18339 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 20:58:25.948132 18339 catalog_manager.cc:1505] Loading token signing keys...
I20250112 20:58:25.962778 18339 catalog_manager.cc:5899] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a: Generated new TSK 0
I20250112 20:58:25.963470 18339 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 20:58:25.982961 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:58:25.990803 18350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:58:25.991959 18351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:25.993253 15454 server_base.cc:1034] running on GCE node
W20250112 20:58:25.994442 18353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:25.995998 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:58:25.996304 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:58:25.996537 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715505996517 us; error 0 us; skew 500 ppm
I20250112 20:58:25.997402 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:58:26.000486 15454 webserver.cc:458] Webserver started at http://127.15.23.129:37837/ using document root <none> and password file <none>
I20250112 20:58:26.001241 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:58:26.001528 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:58:26.001896 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:58:26.003587 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/instance:
uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
format_stamp: "Formatted at 2025-01-12 20:58:25 on dist-test-slave-m2nm"
I20250112 20:58:26.010059 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20250112 20:58:26.013343 18358 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.014026 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250112 20:58:26.014254 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
format_stamp: "Formatted at 2025-01-12 20:58:25 on dist-test-slave-m2nm"
I20250112 20:58:26.014492 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:58:26.045243 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:58:26.046320 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:58:26.047616 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:58:26.049851 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:58:26.050021 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.050268 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:58:26.050418 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.089273 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.129:38493
I20250112 20:58:26.089370 18420 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.129:38493 every 8 connection(s)
I20250112 20:58:26.093756 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:58:26.101598 18425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:58:26.103307 18426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:26.106604 18421 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45335
I20250112 20:58:26.106935 18421 heartbeater.cc:463] Registering TS with master...
W20250112 20:58:26.107275 18429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:26.107635 18421 heartbeater.cc:510] Master 127.15.23.190:45335 requested a full tablet report, sending...
I20250112 20:58:26.108326 15454 server_base.cc:1034] running on GCE node
I20250112 20:58:26.109735 18294 ts_manager.cc:194] Registered new tserver with Master: a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:26.109880 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:58:26.110222 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:58:26.110421 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715506110406 us; error 0 us; skew 500 ppm
I20250112 20:58:26.111167 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:58:26.112001 18294 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43894
I20250112 20:58:26.113981 15454 webserver.cc:458] Webserver started at http://127.15.23.130:37183/ using document root <none> and password file <none>
I20250112 20:58:26.114457 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:58:26.114638 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:58:26.114868 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:58:26.115938 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/instance:
uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
format_stamp: "Formatted at 2025-01-12 20:58:26 on dist-test-slave-m2nm"
I20250112 20:58:26.120057 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250112 20:58:26.123100 18433 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.123812 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250112 20:58:26.124034 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-1-root
uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
format_stamp: "Formatted at 2025-01-12 20:58:26 on dist-test-slave-m2nm"
I20250112 20:58:26.124260 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:58:26.135995 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:58:26.137152 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:58:26.138478 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:58:26.140535 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:58:26.140751 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.140985 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:58:26.141120 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.179157 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.130:35469
I20250112 20:58:26.179260 18495 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.130:35469 every 8 connection(s)
I20250112 20:58:26.183817 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:58:26.190562 18499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:58:26.191947 18500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:26.195642 18496 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45335
I20250112 20:58:26.196108 18496 heartbeater.cc:463] Registering TS with master...
W20250112 20:58:26.196336 18502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:26.196915 18496 heartbeater.cc:510] Master 127.15.23.190:45335 requested a full tablet report, sending...
I20250112 20:58:26.197544 15454 server_base.cc:1034] running on GCE node
I20250112 20:58:26.198432 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:58:26.198638 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:58:26.198805 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715506198788 us; error 0 us; skew 500 ppm
I20250112 20:58:26.198870 18294 ts_manager.cc:194] Registered new tserver with Master: 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:26.199362 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:58:26.200472 18294 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43910
I20250112 20:58:26.202267 15454 webserver.cc:458] Webserver started at http://127.15.23.131:45875/ using document root <none> and password file <none>
I20250112 20:58:26.202948 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:58:26.203162 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:58:26.203379 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:58:26.204438 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/instance:
uuid: "88ef62b247304e079788e24ddfdaa4fb"
format_stamp: "Formatted at 2025-01-12 20:58:26 on dist-test-slave-m2nm"
I20250112 20:58:26.208587 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20250112 20:58:26.211655 18507 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.212383 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250112 20:58:26.212690 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-2-root
uuid: "88ef62b247304e079788e24ddfdaa4fb"
format_stamp: "Formatted at 2025-01-12 20:58:26 on dist-test-slave-m2nm"
I20250112 20:58:26.212949 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:58:26.225647 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:58:26.226881 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:58:26.228291 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:58:26.230415 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:58:26.230615 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.230847 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:58:26.231005 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.268718 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.131:39861
I20250112 20:58:26.268801 18569 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.131:39861 every 8 connection(s)
I20250112 20:58:26.273345 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:58:26.280539 18573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:58:26.282027 18574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:58:26.283151 18576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:26.284454 15454 server_base.cc:1034] running on GCE node
I20250112 20:58:26.285396 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250112 20:58:26.285436 18570 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45335
W20250112 20:58:26.285610 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:58:26.285871 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715506285853 us; error 0 us; skew 500 ppm
I20250112 20:58:26.285888 18570 heartbeater.cc:463] Registering TS with master...
I20250112 20:58:26.286468 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:58:26.286715 18570 heartbeater.cc:510] Master 127.15.23.190:45335 requested a full tablet report, sending...
I20250112 20:58:26.288900 18294 ts_manager.cc:194] Registered new tserver with Master: 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:26.289260 15454 webserver.cc:458] Webserver started at http://127.15.23.132:37997/ using document root <none> and password file <none>
I20250112 20:58:26.289763 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:58:26.289947 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:58:26.290189 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:58:26.290346 18294 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43916
I20250112 20:58:26.291522 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-3-root/instance:
uuid: "c60c0cbe377c450297e3d536b7f7c89c"
format_stamp: "Formatted at 2025-01-12 20:58:26 on dist-test-slave-m2nm"
I20250112 20:58:26.295697 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20250112 20:58:26.299170 18581 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.299861 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250112 20:58:26.300135 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-3-root
uuid: "c60c0cbe377c450297e3d536b7f7c89c"
format_stamp: "Formatted at 2025-01-12 20:58:26 on dist-test-slave-m2nm"
I20250112 20:58:26.300401 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:58:26.315922 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:58:26.317150 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:58:26.318571 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:58:26.320554 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:58:26.320788 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.321014 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:58:26.321167 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:26.361511 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.132:40357
I20250112 20:58:26.361616 18643 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.132:40357 every 8 connection(s)
I20250112 20:58:26.374053 18644 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45335
I20250112 20:58:26.374457 18644 heartbeater.cc:463] Registering TS with master...
I20250112 20:58:26.375174 18644 heartbeater.cc:510] Master 127.15.23.190:45335 requested a full tablet report, sending...
I20250112 20:58:26.376916 18294 ts_manager.cc:194] Registered new tserver with Master: c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:26.377012 15454 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.012310331s
I20250112 20:58:26.378223 18294 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43918
I20250112 20:58:26.379431 15454 test_util.cc:274] Using random seed: -1764398451
I20250112 20:58:26.400997 18294 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43926:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250112 20:58:26.640332 18456 tablet_service.cc:1467] Processing CreateTablet for tablet c257b2d0772f4078b2ca2d7c1298e4ae (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250112 20:58:26.641825 18456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c257b2d0772f4078b2ca2d7c1298e4ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.643383 18385 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3629a889ed427db7d163963dd13507 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250112 20:58:26.645609 18384 tablet_service.cc:1467] Processing CreateTablet for tablet 3d939684f7a845f7b267cf420779b603 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250112 20:58:26.646251 18386 tablet_service.cc:1467] Processing CreateTablet for tablet 4295797580d643d88c1ef2701a123bef (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250112 20:58:26.646716 18454 tablet_service.cc:1467] Processing CreateTablet for tablet f59cd634f5244017b18a111892243f57 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250112 20:58:26.648062 18454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59cd634f5244017b18a111892243f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.647765 18383 tablet_service.cc:1467] Processing CreateTablet for tablet c2f492fad2f5422392d1d141e7534579 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250112 20:58:26.649685 18460 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3629a889ed427db7d163963dd13507 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250112 20:58:26.649919 18382 tablet_service.cc:1467] Processing CreateTablet for tablet d45830b149244e2f8bec3cac5228eb77 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250112 20:58:26.650990 18460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3629a889ed427db7d163963dd13507. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.651836 18381 tablet_service.cc:1467] Processing CreateTablet for tablet eaf48f6743cb46e686eaa798a4a22957 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250112 20:58:26.653774 18457 tablet_service.cc:1467] Processing CreateTablet for tablet d45830b149244e2f8bec3cac5228eb77 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250112 20:58:26.654058 18380 tablet_service.cc:1467] Processing CreateTablet for tablet 9e59ecf931f041feba70ee5460de09da (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250112 20:58:26.655098 18457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45830b149244e2f8bec3cac5228eb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.656010 18379 tablet_service.cc:1467] Processing CreateTablet for tablet f59cd634f5244017b18a111892243f57 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250112 20:58:26.657599 18459 tablet_service.cc:1467] Processing CreateTablet for tablet 1906b96af18e4a7184907ae1b525aa02 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250112 20:58:26.658128 18378 tablet_service.cc:1467] Processing CreateTablet for tablet a5ee370770b54ae593e07378574861d1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250112 20:58:26.658826 18459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1906b96af18e4a7184907ae1b525aa02. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.660166 18377 tablet_service.cc:1467] Processing CreateTablet for tablet 668ccd5a01ca4d4a85a4b033d0dcba06 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250112 20:58:26.661557 18385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3629a889ed427db7d163963dd13507. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.661731 18458 tablet_service.cc:1467] Processing CreateTablet for tablet c2f492fad2f5422392d1d141e7534579 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250112 20:58:26.662205 18377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668ccd5a01ca4d4a85a4b033d0dcba06. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.662796 18381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaf48f6743cb46e686eaa798a4a22957. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.662963 18458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f492fad2f5422392d1d141e7534579. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.663380 18378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ee370770b54ae593e07378574861d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.664772 18384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d939684f7a845f7b267cf420779b603. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.665831 18379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59cd634f5244017b18a111892243f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.665761 18461 tablet_service.cc:1467] Processing CreateTablet for tablet 4295797580d643d88c1ef2701a123bef (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250112 20:58:26.667098 18461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4295797580d643d88c1ef2701a123bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.667219 18383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f492fad2f5422392d1d141e7534579. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.669894 18453 tablet_service.cc:1467] Processing CreateTablet for tablet 5547898f3d53475fb47696f4ddd6ef1c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250112 20:58:26.671546 18453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5547898f3d53475fb47696f4ddd6ef1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.672147 18386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4295797580d643d88c1ef2701a123bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.645609 18455 tablet_service.cc:1467] Processing CreateTablet for tablet 9e59ecf931f041feba70ee5460de09da (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250112 20:58:26.673192 18382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45830b149244e2f8bec3cac5228eb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.673630 18455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e59ecf931f041feba70ee5460de09da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.674243 18380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e59ecf931f041feba70ee5460de09da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.701687 18452 tablet_service.cc:1467] Processing CreateTablet for tablet 668ccd5a01ca4d4a85a4b033d0dcba06 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250112 20:58:26.720927 18452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668ccd5a01ca4d4a85a4b033d0dcba06. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.750092 18535 tablet_service.cc:1467] Processing CreateTablet for tablet 4295797580d643d88c1ef2701a123bef (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION VALUES < 36398027
I20250112 20:58:26.751528 18535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4295797580d643d88c1ef2701a123bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.767112 18384 tablet_service.cc:1467] Processing CreateTablet for tablet 1b92c85eeea5465ea50fa57eb9214098 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250112 20:58:26.768611 18384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b92c85eeea5465ea50fa57eb9214098. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.782830 18606 tablet_service.cc:1467] Processing CreateTablet for tablet d45830b149244e2f8bec3cac5228eb77 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20250112 20:58:26.784178 18605 tablet_service.cc:1467] Processing CreateTablet for tablet eaf48f6743cb46e686eaa798a4a22957 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250112 20:58:26.791847 18602 tablet_service.cc:1467] Processing CreateTablet for tablet a5ee370770b54ae593e07378574861d1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250112 20:58:26.789733 18533 tablet_service.cc:1467] Processing CreateTablet for tablet 1906b96af18e4a7184907ae1b525aa02 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250112 20:58:26.790849 18532 tablet_service.cc:1467] Processing CreateTablet for tablet c2f492fad2f5422392d1d141e7534579 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20250112 20:58:26.790808 18603 tablet_service.cc:1467] Processing CreateTablet for tablet f59cd634f5244017b18a111892243f57 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20250112 20:58:26.787302 18607 tablet_service.cc:1467] Processing CreateTablet for tablet 1906b96af18e4a7184907ae1b525aa02 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20250112 20:58:26.795364 18531 tablet_service.cc:1467] Processing CreateTablet for tablet eaf48f6743cb46e686eaa798a4a22957 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20250112 20:58:26.789554 18604 tablet_service.cc:1467] Processing CreateTablet for tablet c257b2d0772f4078b2ca2d7c1298e4ae (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250112 20:58:26.785269 18606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d45830b149244e2f8bec3cac5228eb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.784010 18608 tablet_service.cc:1467] Processing CreateTablet for tablet 3d939684f7a845f7b267cf420779b603 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250112 20:58:26.785382 18661 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:26.796557 18607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1906b96af18e4a7184907ae1b525aa02. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.795153 18533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1906b96af18e4a7184907ae1b525aa02. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.797308 18608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d939684f7a845f7b267cf420779b603. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.791038 18534 tablet_service.cc:1467] Processing CreateTablet for tablet 3d939684f7a845f7b267cf420779b603 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20250112 20:58:26.797973 18605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaf48f6743cb46e686eaa798a4a22957. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.782819 18609 tablet_service.cc:1467] Processing CreateTablet for tablet 6f3629a889ed427db7d163963dd13507 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20250112 20:58:26.798650 18602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ee370770b54ae593e07378574861d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.802899 18532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f492fad2f5422392d1d141e7534579. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.804299 18534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d939684f7a845f7b267cf420779b603. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.804332 18530 tablet_service.cc:1467] Processing CreateTablet for tablet c257b2d0772f4078b2ca2d7c1298e4ae (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20250112 20:58:26.805824 18530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c257b2d0772f4078b2ca2d7c1298e4ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.809047 18531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaf48f6743cb46e686eaa798a4a22957. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.808913 18461 tablet_service.cc:1467] Processing CreateTablet for tablet ef6e68976f1b46278833369d2b5c0e95 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250112 20:58:26.809803 18529 tablet_service.cc:1467] Processing CreateTablet for tablet 9e59ecf931f041feba70ee5460de09da (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20250112 20:58:26.810496 18600 tablet_service.cc:1467] Processing CreateTablet for tablet 1b92c85eeea5465ea50fa57eb9214098 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250112 20:58:26.810473 18461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6e68976f1b46278833369d2b5c0e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.811981 18528 tablet_service.cc:1467] Processing CreateTablet for tablet a5ee370770b54ae593e07378574861d1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20250112 20:58:26.813948 18453 tablet_service.cc:1467] Processing CreateTablet for tablet ee08d81c32b648e085f17dd937f5c512 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250112 20:58:26.811604 18601 tablet_service.cc:1467] Processing CreateTablet for tablet 5547898f3d53475fb47696f4ddd6ef1c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250112 20:58:26.815039 18528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5ee370770b54ae593e07378574861d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.815423 18453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee08d81c32b648e085f17dd937f5c512. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.816913 18529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e59ecf931f041feba70ee5460de09da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.819051 18603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f59cd634f5244017b18a111892243f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.819828 18609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3629a889ed427db7d163963dd13507. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.821481 18601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5547898f3d53475fb47696f4ddd6ef1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.822088 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b92c85eeea5465ea50fa57eb9214098. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.824967 18604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c257b2d0772f4078b2ca2d7c1298e4ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.830371 18386 tablet_service.cc:1467] Processing CreateTablet for tablet ee08d81c32b648e085f17dd937f5c512 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250112 20:58:26.831729 18386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee08d81c32b648e085f17dd937f5c512. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.833808 18378 tablet_service.cc:1467] Processing CreateTablet for tablet bb32432c5bc044b5b9f6871bbb30be64 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250112 20:58:26.837234 18382 tablet_service.cc:1467] Processing CreateTablet for tablet 8db962a176db442490ad2e6133babb67 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250112 20:58:26.787919 18459 tablet_service.cc:1467] Processing CreateTablet for tablet 1b92c85eeea5465ea50fa57eb9214098 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20250112 20:58:26.853708 18459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b92c85eeea5465ea50fa57eb9214098. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.853981 18379 tablet_service.cc:1467] Processing CreateTablet for tablet 4b0cf559505640aea4edf6feedf96817 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250112 20:58:26.864908 18455 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec5696903d64121bab105da9eb5a3b9 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250112 20:58:26.866114 18455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec5696903d64121bab105da9eb5a3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.869287 18460 tablet_service.cc:1467] Processing CreateTablet for tablet bb32432c5bc044b5b9f6871bbb30be64 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250112 20:58:26.870504 18460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb32432c5bc044b5b9f6871bbb30be64. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.872519 18379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0cf559505640aea4edf6feedf96817. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.872648 18600 tablet_service.cc:1467] Processing CreateTablet for tablet 8db962a176db442490ad2e6133babb67 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250112 20:58:26.877774 18385 tablet_service.cc:1467] Processing CreateTablet for tablet 6df407527b0d446795e620dec471e027 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250112 20:58:26.879011 18385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6df407527b0d446795e620dec471e027. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.879704 18527 tablet_service.cc:1467] Processing CreateTablet for tablet 5547898f3d53475fb47696f4ddd6ef1c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20250112 20:58:26.880770 18378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb32432c5bc044b5b9f6871bbb30be64. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.880954 18527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5547898f3d53475fb47696f4ddd6ef1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.888965 18526 tablet_service.cc:1467] Processing CreateTablet for tablet 668ccd5a01ca4d4a85a4b033d0dcba06 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20250112 20:58:26.890357 18526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 668ccd5a01ca4d4a85a4b033d0dcba06. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.889333 18382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db962a176db442490ad2e6133babb67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.875559 18381 tablet_service.cc:1467] Processing CreateTablet for tablet e1f0a2bc717d404f8f3afd03c3d5a60c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250112 20:58:26.892024 18381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f0a2bc717d404f8f3afd03c3d5a60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.902545 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db962a176db442490ad2e6133babb67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.909822 18662 tablet_bootstrap.cc:492] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
W20250112 20:58:26.911770 18342 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:58:26.947681 18661 tablet_bootstrap.cc:700] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent opening tablet: real 0.108s user 0.004s sys 0.000s
I20250112 20:58:26.948542 18661 tablet_bootstrap.cc:654] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:26.989871 18663 tablet_bootstrap.cc:492] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:26.995234 18603 tablet_service.cc:1467] Processing CreateTablet for tablet ef6e68976f1b46278833369d2b5c0e95 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250112 20:58:27.013926 18603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6e68976f1b46278833369d2b5c0e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.020054 18663 tablet_bootstrap.cc:654] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:27.022264 18604 tablet_service.cc:1467] Processing CreateTablet for tablet bb32432c5bc044b5b9f6871bbb30be64 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20250112 20:58:27.023597 18604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb32432c5bc044b5b9f6871bbb30be64. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:58:27.020390 18279 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 90987us
I20250112 20:58:26.944406 18454 tablet_service.cc:1467] Processing CreateTablet for tablet e1f0a2bc717d404f8f3afd03c3d5a60c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250112 20:58:27.028560 18454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f0a2bc717d404f8f3afd03c3d5a60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:26.995765 18457 tablet_service.cc:1467] Processing CreateTablet for tablet 6df407527b0d446795e620dec471e027 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250112 20:58:27.031502 18457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6df407527b0d446795e620dec471e027. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.000496 18380 tablet_service.cc:1467] Processing CreateTablet for tablet b7f8bbae0e0c494ab580a5f2338ef4ea (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250112 20:58:27.036361 18380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7f8bbae0e0c494ab580a5f2338ef4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.039717 18662 tablet_bootstrap.cc:654] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:27.047364 18605 tablet_service.cc:1467] Processing CreateTablet for tablet 4b0cf559505640aea4edf6feedf96817 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250112 20:58:27.050649 18528 tablet_service.cc:1467] Processing CreateTablet for tablet 8db962a176db442490ad2e6133babb67 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20250112 20:58:27.057979 18384 tablet_service.cc:1467] Processing CreateTablet for tablet f7f1fd690e5d4207957654f28756dfaf (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250112 20:58:27.063640 18535 tablet_service.cc:1467] Processing CreateTablet for tablet ef6e68976f1b46278833369d2b5c0e95 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20250112 20:58:27.066807 18377 tablet_service.cc:1467] Processing CreateTablet for tablet 013cadee88a64ffe9b1b31198139d6d1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250112 20:58:27.067907 18663 tablet_bootstrap.cc:492] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:27.145037 18663 ts_tablet_manager.cc:1397] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.156s user 0.004s sys 0.007s
I20250112 20:58:27.075244 18381 tablet_service.cc:1467] Processing CreateTablet for tablet 7b6dc91797f14584b833e9dec80369f5 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250112 20:58:27.115028 18607 tablet_service.cc:1467] Processing CreateTablet for tablet f7f1fd690e5d4207957654f28756dfaf (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250112 20:58:27.148283 18382 tablet_service.cc:1467] Processing CreateTablet for tablet c99e10fcc8944b44aff69f16d268a07c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250112 20:58:27.149125 18663 raft_consensus.cc:357] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:27.081189 18461 tablet_service.cc:1467] Processing CreateTablet for tablet b7f8bbae0e0c494ab580a5f2338ef4ea (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250112 20:58:27.150840 18461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7f8bbae0e0c494ab580a5f2338ef4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.151207 18532 tablet_service.cc:1467] Processing CreateTablet for tablet b7f8bbae0e0c494ab580a5f2338ef4ea (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20250112 20:58:27.152837 18534 tablet_service.cc:1467] Processing CreateTablet for tablet e1f0a2bc717d404f8f3afd03c3d5a60c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20250112 20:58:27.153045 18385 tablet_service.cc:1467] Processing CreateTablet for tablet 1332a302b6a44f9c8963dfed69c078a5 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250112 20:58:27.081895 18606 tablet_service.cc:1467] Processing CreateTablet for tablet 684a3c0a345f4ce483768dfe1572ca13 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250112 20:58:27.082960 18531 tablet_service.cc:1467] Processing CreateTablet for tablet ee08d81c32b648e085f17dd937f5c512 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20250112 20:58:27.087457 18379 tablet_service.cc:1467] Processing CreateTablet for tablet 737c8961a9424363b63a9688853bea1a (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250112 20:58:27.156916 18663 raft_consensus.cc:383] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:27.096589 18384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f1fd690e5d4207957654f28756dfaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.088029 18527 tablet_service.cc:1467] Processing CreateTablet for tablet 4b0cf559505640aea4edf6feedf96817 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20250112 20:58:27.098953 18600 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec5696903d64121bab105da9eb5a3b9 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250112 20:58:27.102895 18529 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec5696903d64121bab105da9eb5a3b9 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20250112 20:58:27.081189 18458 tablet_service.cc:1467] Processing CreateTablet for tablet 3bea0704826246c1b1e2901dbab0cb15 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250112 20:58:27.106613 18386 tablet_service.cc:1467] Processing CreateTablet for tablet fb4e64edf27d4e91aca4fba312aff693 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250112 20:58:27.109424 18605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0cf559505640aea4edf6feedf96817. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.109899 18456 tablet_service.cc:1467] Processing CreateTablet for tablet f7f1fd690e5d4207957654f28756dfaf (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20250112 20:58:27.110977 18602 tablet_service.cc:1467] Processing CreateTablet for tablet 3bea0704826246c1b1e2901dbab0cb15 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250112 20:58:27.114809 18421 heartbeater.cc:502] Master 127.15.23.190:45335 was elected leader, sending a full tablet report...
I20250112 20:58:27.116341 18526 tablet_service.cc:1467] Processing CreateTablet for tablet 684a3c0a345f4ce483768dfe1572ca13 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250112 20:58:27.121189 18528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db962a176db442490ad2e6133babb67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.121688 18604 tablet_service.cc:1467] Processing CreateTablet for tablet 013cadee88a64ffe9b1b31198139d6d1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250112 20:58:27.165620 18386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb4e64edf27d4e91aca4fba312aff693. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.131181 18530 tablet_service.cc:1467] Processing CreateTablet for tablet 3bea0704826246c1b1e2901dbab0cb15 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20250112 20:58:27.134690 18452 tablet_service.cc:1467] Processing CreateTablet for tablet eeb9028573e647bba81035b9c4a15146 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250112 20:58:26.999614 18383 tablet_service.cc:1467] Processing CreateTablet for tablet 684a3c0a345f4ce483768dfe1572ca13 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20250112 20:58:27.156347 18380 tablet_service.cc:1467] Processing CreateTablet for tablet af0c89ea5b0d4655959ccaad0b838bf4 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250112 20:58:27.157613 18663 raft_consensus.cc:738] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:27.173427 18379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737c8961a9424363b63a9688853bea1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.174664 18533 tablet_service.cc:1467] Processing CreateTablet for tablet 013cadee88a64ffe9b1b31198139d6d1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20250112 20:58:27.176198 18606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684a3c0a345f4ce483768dfe1572ca13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.176784 18381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b6dc91797f14584b833e9dec80369f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.177421 18455 tablet_service.cc:1467] Processing CreateTablet for tablet 737c8961a9424363b63a9688853bea1a (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250112 20:58:27.180903 18382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99e10fcc8944b44aff69f16d268a07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.180910 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec5696903d64121bab105da9eb5a3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.181927 18661 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:27.182420 18661 ts_tablet_manager.cc:1397] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.416s user 0.012s sys 0.000s
I20250112 20:58:27.184934 18602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bea0704826246c1b1e2901dbab0cb15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.184986 18377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013cadee88a64ffe9b1b31198139d6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.187487 18385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1332a302b6a44f9c8963dfed69c078a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.190809 18383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684a3c0a345f4ce483768dfe1572ca13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.197770 18458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bea0704826246c1b1e2901dbab0cb15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.198050 18664 tablet_bootstrap.cc:492] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:27.200734 18604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013cadee88a64ffe9b1b31198139d6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.203305 18496 heartbeater.cc:502] Master 127.15.23.190:45335 was elected leader, sending a full tablet report...
I20250112 20:58:27.208791 18609 tablet_service.cc:1467] Processing CreateTablet for tablet eeb9028573e647bba81035b9c4a15146 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250112 20:58:27.210196 18609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb9028573e647bba81035b9c4a15146. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.212818 18608 tablet_service.cc:1467] Processing CreateTablet for tablet 6df407527b0d446795e620dec471e027 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20250112 20:58:27.213155 18661 raft_consensus.cc:357] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:27.213944 18661 raft_consensus.cc:383] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:27.214123 18608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6df407527b0d446795e620dec471e027. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.214272 18661 raft_consensus.cc:738] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:27.215169 18661 consensus_queue.cc:260] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:27.216388 18663 consensus_queue.cc:260] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:27.216689 18601 tablet_service.cc:1467] Processing CreateTablet for tablet fb4e64edf27d4e91aca4fba312aff693 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20250112 20:58:27.217922 18601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb4e64edf27d4e91aca4fba312aff693. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.221650 18382 tablet_service.cc:1467] Processing CreateTablet for tablet 60856bd5ad724e3792e927130b6315cf (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250112 20:58:27.222878 18382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60856bd5ad724e3792e927130b6315cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.227922 18531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee08d81c32b648e085f17dd937f5c512. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.228834 18378 tablet_service.cc:1467] Processing CreateTablet for tablet 523dfa2a1c9e4cdf9def87c077e7c7f1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250112 20:58:27.235927 18378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523dfa2a1c9e4cdf9def87c077e7c7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.241401 18459 tablet_service.cc:1467] Processing CreateTablet for tablet 7b6dc91797f14584b833e9dec80369f5 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250112 20:58:27.242303 18603 tablet_service.cc:1467] Processing CreateTablet for tablet 737c8961a9424363b63a9688853bea1a (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20250112 20:58:27.243541 18603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737c8961a9424363b63a9688853bea1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.244937 18527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b0cf559505640aea4edf6feedf96817. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.245899 18453 tablet_service.cc:1467] Processing CreateTablet for tablet 8300b885639841769a76417fb45f2db7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250112 20:58:27.234760 18662 tablet_bootstrap.cc:492] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:27.247212 18662 ts_tablet_manager.cc:1397] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.338s user 0.010s sys 0.028s
I20250112 20:58:27.248924 18533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013cadee88a64ffe9b1b31198139d6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.250085 18662 raft_consensus.cc:357] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:27.250955 18662 raft_consensus.cc:383] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:27.251384 18526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684a3c0a345f4ce483768dfe1572ca13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.251302 18662 raft_consensus.cc:738] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:27.252899 18529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ec5696903d64121bab105da9eb5a3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.255810 18600 tablet_service.cc:1467] Processing CreateTablet for tablet 8300b885639841769a76417fb45f2db7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250112 20:58:27.256893 18535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6e68976f1b46278833369d2b5c0e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.257366 18456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f1fd690e5d4207957654f28756dfaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.257809 18607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7f1fd690e5d4207957654f28756dfaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.259271 18381 tablet_service.cc:1467] Processing CreateTablet for tablet 16d5dc78c20741968c7ca8f7ecec1c07 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250112 20:58:27.281186 18532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7f8bbae0e0c494ab580a5f2338ef4ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.284909 18453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8300b885639841769a76417fb45f2db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.288889 18454 tablet_service.cc:1467] Processing CreateTablet for tablet c99e10fcc8944b44aff69f16d268a07c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250112 20:58:27.290206 18454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99e10fcc8944b44aff69f16d268a07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.292523 18570 heartbeater.cc:502] Master 127.15.23.190:45335 was elected leader, sending a full tablet report...
I20250112 20:58:27.300120 18534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1f0a2bc717d404f8f3afd03c3d5a60c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.301010 18452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb9028573e647bba81035b9c4a15146. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.301788 18530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bea0704826246c1b1e2901dbab0cb15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.303331 18455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737c8961a9424363b63a9688853bea1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.320614 18459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b6dc91797f14584b833e9dec80369f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.328487 18386 tablet_service.cc:1467] Processing CreateTablet for tablet 08844b71d8c54556af7a5d625ed646c2 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250112 20:58:27.329222 18377 tablet_service.cc:1467] Processing CreateTablet for tablet 985f9b21969e48739677e3cdc86e25de (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250112 20:58:27.330485 18663 ts_tablet_manager.cc:1428] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.185s user 0.015s sys 0.030s
I20250112 20:58:27.324925 18382 tablet_service.cc:1467] Processing CreateTablet for tablet 219a31736df4499fb96cacfe05747d1a (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250112 20:58:27.331236 18644 heartbeater.cc:502] Master 127.15.23.190:45335 was elected leader, sending a full tablet report...
I20250112 20:58:27.326869 18384 tablet_service.cc:1467] Processing CreateTablet for tablet f8820dfc60bc477985f861406c22dd3b (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250112 20:58:27.332341 18381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d5dc78c20741968c7ca8f7ecec1c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.336961 18384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8820dfc60bc477985f861406c22dd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.337651 18457 tablet_service.cc:1467] Processing CreateTablet for tablet 759cdb36c54b4950b89e7502ae9e878f (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250112 20:58:27.338842 18457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759cdb36c54b4950b89e7502ae9e878f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.319415 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8300b885639841769a76417fb45f2db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.344254 18460 tablet_service.cc:1467] Processing CreateTablet for tablet 60856bd5ad724e3792e927130b6315cf (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250112 20:58:27.344961 18382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219a31736df4499fb96cacfe05747d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.345690 18380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af0c89ea5b0d4655959ccaad0b838bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.345721 18663 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:27.352325 18386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08844b71d8c54556af7a5d625ed646c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.345675 18460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60856bd5ad724e3792e927130b6315cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.357451 18377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985f9b21969e48739677e3cdc86e25de. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:58:27.366048 18645 tablet.cc:2367] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 20:58:27.366410 18385 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa320ac4d374bcc8a1af58dc71bcdfe (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250112 20:58:27.368191 18385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa320ac4d374bcc8a1af58dc71bcdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.408088 18663 tablet_bootstrap.cc:654] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:27.320170 18662 consensus_queue.cc:260] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:27.421710 18669 raft_consensus.cc:491] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:27.426442 18669 raft_consensus.cc:513] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:27.429164 18609 tablet_service.cc:1467] Processing CreateTablet for tablet 1332a302b6a44f9c8963dfed69c078a5 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250112 20:58:27.430454 18609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1332a302b6a44f9c8963dfed69c078a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:58:27.425186 18279 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 59656.3us
I20250112 20:58:27.458983 18669 leader_election.cc:290] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:27.459672 18529 tablet_service.cc:1467] Processing CreateTablet for tablet 7b6dc91797f14584b833e9dec80369f5 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20250112 20:58:27.479883 18606 tablet_service.cc:1467] Processing CreateTablet for tablet af0c89ea5b0d4655959ccaad0b838bf4 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250112 20:58:27.482448 18606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af0c89ea5b0d4655959ccaad0b838bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.489619 18386 tablet_service.cc:1467] Processing CreateTablet for tablet afe68bc3c3a54ebb816b85651cfd9d08 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250112 20:58:27.481132 18461 tablet_service.cc:1467] Processing CreateTablet for tablet 1332a302b6a44f9c8963dfed69c078a5 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20250112 20:58:27.482831 18605 tablet_service.cc:1467] Processing CreateTablet for tablet 759cdb36c54b4950b89e7502ae9e878f (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250112 20:58:27.490926 18386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe68bc3c3a54ebb816b85651cfd9d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.491089 18605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759cdb36c54b4950b89e7502ae9e878f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.494042 18609 tablet_service.cc:1467] Processing CreateTablet for tablet 16d5dc78c20741968c7ca8f7ecec1c07 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250112 20:58:27.495273 18609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d5dc78c20741968c7ca8f7ecec1c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.494987 18608 tablet_service.cc:1467] Processing CreateTablet for tablet fe90092ab277453fb41e05a8be798304 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250112 20:58:27.496203 18608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe90092ab277453fb41e05a8be798304. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.497391 18461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1332a302b6a44f9c8963dfed69c078a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.503756 18381 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca1ca0f378247598c915c11faee8ed2 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250112 20:58:27.415367 18379 tablet_service.cc:1467] Processing CreateTablet for tablet 3f52ac1efd0e417393e5127713a43a18 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250112 20:58:27.505048 18381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca1ca0f378247598c915c11faee8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.517196 18379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f52ac1efd0e417393e5127713a43a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.488269 18601 tablet_service.cc:1467] Processing CreateTablet for tablet 523dfa2a1c9e4cdf9def87c077e7c7f1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250112 20:58:27.519562 18601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523dfa2a1c9e4cdf9def87c077e7c7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.498690 18603 tablet_service.cc:1467] Processing CreateTablet for tablet 219a31736df4499fb96cacfe05747d1a (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250112 20:58:27.512284 18602 tablet_service.cc:1467] Processing CreateTablet for tablet 161a73359f3a4bb19a4df8ad30c6669e (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250112 20:58:27.527451 18456 tablet_service.cc:1467] Processing CreateTablet for tablet fe90092ab277453fb41e05a8be798304 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250112 20:58:27.528764 18456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe90092ab277453fb41e05a8be798304. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.531232 18602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161a73359f3a4bb19a4df8ad30c6669e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.533192 18605 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa320ac4d374bcc8a1af58dc71bcdfe (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250112 20:58:27.538887 18605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa320ac4d374bcc8a1af58dc71bcdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.536720 18664 tablet_bootstrap.cc:700] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb: Time spent opening tablet: real 0.101s user 0.004s sys 0.000s
I20250112 20:58:27.545217 18664 tablet_bootstrap.cc:654] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:27.535291 18378 tablet_service.cc:1467] Processing CreateTablet for tablet d7daf053c1144c6797f1ac89fedd0fd0 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250112 20:58:27.547509 18378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7daf053c1144c6797f1ac89fedd0fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.556396 18459 tablet_service.cc:1467] Processing CreateTablet for tablet 985f9b21969e48739677e3cdc86e25de (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250112 20:58:27.558071 18459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985f9b21969e48739677e3cdc86e25de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.442682 18661 ts_tablet_manager.cc:1428] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.260s user 0.015s sys 0.013s
I20250112 20:58:27.561340 18661 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:27.465735 18526 tablet_service.cc:1467] Processing CreateTablet for tablet eeb9028573e647bba81035b9c4a15146 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20250112 20:58:27.584345 18529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b6dc91797f14584b833e9dec80369f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.585261 18528 tablet_service.cc:1467] Processing CreateTablet for tablet afe68bc3c3a54ebb816b85651cfd9d08 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250112 20:58:27.580963 18604 tablet_service.cc:1467] Processing CreateTablet for tablet f8820dfc60bc477985f861406c22dd3b (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250112 20:58:27.586935 18662 ts_tablet_manager.cc:1428] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.339s user 0.015s sys 0.014s
I20250112 20:58:27.587992 18662 tablet_bootstrap.cc:492] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:27.587747 18533 tablet_service.cc:1467] Processing CreateTablet for tablet fb4e64edf27d4e91aca4fba312aff693 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
W20250112 20:58:27.577608 18279 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 54711.2us
W20250112 20:58:27.620332 18281 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 56805.2us
I20250112 20:58:27.621731 18670 raft_consensus.cc:491] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:27.625202 18670 raft_consensus.cc:513] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:27.626317 18668 raft_consensus.cc:491] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:27.626821 18668 raft_consensus.cc:513] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:27.512235 18458 tablet_service.cc:1467] Processing CreateTablet for tablet 523dfa2a1c9e4cdf9def87c077e7c7f1 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20250112 20:58:27.620854 18603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219a31736df4499fb96cacfe05747d1a. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:58:27.644418 18280 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.131:39861, user_credentials={real_user=slave}} blocked reactor thread for 51123.2us
I20250112 20:58:27.651026 18528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe68bc3c3a54ebb816b85651cfd9d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.653182 18533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb4e64edf27d4e91aca4fba312aff693. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.653795 18527 tablet_service.cc:1467] Processing CreateTablet for tablet 8300b885639841769a76417fb45f2db7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20250112 20:58:27.655079 18527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8300b885639841769a76417fb45f2db7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.657112 18526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb9028573e647bba81035b9c4a15146. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.659274 18382 tablet_service.cc:1467] Processing CreateTablet for tablet cdbd6c5512664ac7a36a13bd1a89d75f (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250112 20:58:27.661039 18668 leader_election.cc:290] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:27.634231 18670 leader_election.cc:290] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:27.660538 18382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdbd6c5512664ac7a36a13bd1a89d75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.680922 18604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8820dfc60bc477985f861406c22dd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.705823 18600 tablet_service.cc:1467] Processing CreateTablet for tablet 3f52ac1efd0e417393e5127713a43a18 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250112 20:58:27.707171 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f52ac1efd0e417393e5127713a43a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.716095 18452 tablet_service.cc:1467] Processing CreateTablet for tablet 161a73359f3a4bb19a4df8ad30c6669e (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250112 20:58:27.716621 18453 tablet_service.cc:1467] Processing CreateTablet for tablet 08844b71d8c54556af7a5d625ed646c2 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250112 20:58:27.718408 18661 tablet_bootstrap.cc:654] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:27.721901 18600 tablet_service.cc:1467] Processing CreateTablet for tablet 6db2806263c3471cac0496b194a73ab9 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250112 20:58:27.722832 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db2806263c3471cac0496b194a73ab9. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:58:27.704069 18280 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.131:39861, user_credentials={real_user=slave}} blocked reactor thread for 54677.8us
I20250112 20:58:27.705865 18385 tablet_service.cc:1467] Processing CreateTablet for tablet 6961f63ec790480aaa86f70431ed86c6 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250112 20:58:27.737008 18600 tablet_service.cc:1467] Processing CreateTablet for tablet d7daf053c1144c6797f1ac89fedd0fd0 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250112 20:58:27.737847 18385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6961f63ec790480aaa86f70431ed86c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.738189 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7daf053c1144c6797f1ac89fedd0fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.739876 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5547898f3d53475fb47696f4ddd6ef1c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:27.741668 18436 leader_election.cc:343] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:27.746536 18535 tablet_service.cc:1467] Processing CreateTablet for tablet af0c89ea5b0d4655959ccaad0b838bf4 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20250112 20:58:27.747815 18535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af0c89ea5b0d4655959ccaad0b838bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.754137 18600 tablet_service.cc:1467] Processing CreateTablet for tablet e8704b5088604084a641aa00baf84be8 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250112 20:58:27.755523 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8704b5088604084a641aa00baf84be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.755995 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d939684f7a845f7b267cf420779b603" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:27.757464 18361 leader_election.cc:343] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:27.754254 18454 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa320ac4d374bcc8a1af58dc71bcdfe (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20250112 20:58:27.766029 18383 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57802 (request call id 17) took 1025 ms. Trace:
I20250112 20:58:27.766455 18383 rpcz_store.cc:276] 0112 20:58:26.740338 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:26.998712 (+258374us) service_pool.cc:225] Handling call
0112 20:58:27.172542 (+173830us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.190464 (+ 17922us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.322922 (+132458us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:27.765558 (+442636us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:27.765974 (+ 416us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:27.768576 18383 tablet_service.cc:1467] Processing CreateTablet for tablet 4bed5fc535ff4870870d2b8f20d637d7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250112 20:58:27.769438 18455 tablet_service.cc:1467] Processing CreateTablet for tablet 219a31736df4499fb96cacfe05747d1a (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20250112 20:58:27.769882 18383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bed5fc535ff4870870d2b8f20d637d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.737776 18532 tablet_service.cc:1467] Processing CreateTablet for tablet c99e10fcc8944b44aff69f16d268a07c (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20250112 20:58:27.773350 18532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c99e10fcc8944b44aff69f16d268a07c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.774298 18600 tablet_service.cc:1467] Processing CreateTablet for tablet 4bed5fc535ff4870870d2b8f20d637d7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250112 20:58:27.775887 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bed5fc535ff4870870d2b8f20d637d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.777391 18460 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca1ca0f378247598c915c11faee8ed2 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20250112 20:58:27.782629 18662 tablet_bootstrap.cc:654] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:27.702365 18663 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:27.793071 18663 ts_tablet_manager.cc:1397] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.449s user 0.017s sys 0.015s
I20250112 20:58:27.793447 18380 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57802 (request call id 26) took 1034 ms. Trace:
I20250112 20:58:27.793859 18380 rpcz_store.cc:276] 0112 20:58:26.758756 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.155502 (+396746us) service_pool.cc:225] Handling call
0112 20:58:27.345110 (+189608us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b58001e0398. stack: 00007fad7938b853 00007fad7938b750 00007fad78cdd584 00007fad827d2ff4 00007fad827d2f89 00007fad792012d1 00007fad79380291 00007fad79380142 00007fad82050cd9 00007fad811df750 00007fad812c1d91 00007fad8129f243 00007fad81250859 00007fad7a836c64 00007fad7a836bf2 00007fad7a836b21
0112 20:58:27.345262 (+ 152us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.345343 (+ 81us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.732769 (+387426us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:27.792953 (+ 60184us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:27.793389 (+ 436us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5398784}
I20250112 20:58:27.795854 18600 tablet_service.cc:1467] Processing CreateTablet for tablet e73763ba5b1a4f59901b9e8f5eb6ab78 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250112 20:58:27.797559 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73763ba5b1a4f59901b9e8f5eb6ab78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.702558 18458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 523dfa2a1c9e4cdf9def87c077e7c7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.736057 18606 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca1ca0f378247598c915c11faee8ed2 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20250112 20:58:27.686901 18281 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 43814.5us
I20250112 20:58:27.795938 18663 raft_consensus.cc:357] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:27.820024 18663 raft_consensus.cc:383] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:27.833316 18600 tablet_service.cc:1467] Processing CreateTablet for tablet 95e82471468e430b877403ce304eac78 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250112 20:58:27.834216 18600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95e82471468e430b877403ce304eac78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.839190 18461 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 25) took 1012 ms. Trace:
I20250112 20:58:27.754202 18607 tablet_service.cc:1467] Processing CreateTablet for tablet 6961f63ec790480aaa86f70431ed86c6 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250112 20:58:27.841763 18384 tablet_service.cc:1467] Processing CreateTablet for tablet f4bdf5e8ae124786b98df6013ed28460 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250112 20:58:27.848273 18384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdf5e8ae124786b98df6013ed28460. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.848452 18531 tablet_service.cc:1467] Processing CreateTablet for tablet 759cdb36c54b4950b89e7502ae9e878f (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20250112 20:58:27.846671 18661 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:27.853106 18661 ts_tablet_manager.cc:1397] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.292s user 0.016s sys 0.016s
I20250112 20:58:27.855720 18661 raft_consensus.cc:357] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
W20250112 20:58:27.815366 18280 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.131:39861, user_credentials={real_user=slave}} blocked reactor thread for 90076.1us
I20250112 20:58:27.830266 18454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fa320ac4d374bcc8a1af58dc71bcdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.856416 18661 raft_consensus.cc:383] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:27.865116 18461 rpcz_store.cc:276] 0112 20:58:26.826530 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.316498 (+489968us) service_pool.cc:225] Handling call
0112 20:58:27.496965 (+180467us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.497052 (+ 87us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.757386 (+260334us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:27.838730 (+ 81344us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:27.839128 (+ 398us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:27.872817 18531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 759cdb36c54b4950b89e7502ae9e878f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.873553 18661 raft_consensus.cc:738] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:27.879081 18526 tablet_service.cc:1467] Processing CreateTablet for tablet 60856bd5ad724e3792e927130b6315cf (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20250112 20:58:27.881229 18386 tablet_service.cc:1467] Processing CreateTablet for tablet 95e82471468e430b877403ce304eac78 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250112 20:58:27.883479 18386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95e82471468e430b877403ce304eac78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.884238 18528 tablet_service.cc:1467] Processing CreateTablet for tablet 16d5dc78c20741968c7ca8f7ecec1c07 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20250112 20:58:27.881439 18526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60856bd5ad724e3792e927130b6315cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.886840 18528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16d5dc78c20741968c7ca8f7ecec1c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.880569 18461 tablet_service.cc:1467] Processing CreateTablet for tablet afe68bc3c3a54ebb816b85651cfd9d08 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20250112 20:58:27.898976 18606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca1ca0f378247598c915c11faee8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.835789 18663 raft_consensus.cc:738] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:27.901535 18530 tablet_service.cc:1467] Processing CreateTablet for tablet fe90092ab277453fb41e05a8be798304 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20250112 20:58:27.907040 18530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe90092ab277453fb41e05a8be798304. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.916092 18607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6961f63ec790480aaa86f70431ed86c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.922930 18380 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250112 20:58:27.924418 18380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc3d5bd7d04459c9c1e7ccb28eb2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.928365 18452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161a73359f3a4bb19a4df8ad30c6669e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.930321 18453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08844b71d8c54556af7a5d625ed646c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.933790 18609 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250112 20:58:27.935328 18609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc3d5bd7d04459c9c1e7ccb28eb2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.938889 18457 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 26) took 1111 ms. Trace:
I20250112 20:58:27.939414 18457 rpcz_store.cc:276] 0112 20:58:26.826823 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.336794 (+509971us) service_pool.cc:225] Handling call
0112 20:58:27.338421 (+ 1627us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.338510 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.761377 (+422867us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:27.937640 (+176263us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:27.938210 (+ 570us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:27.941800 18457 tablet_service.cc:1467] Processing CreateTablet for tablet 6db2806263c3471cac0496b194a73ab9 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250112 20:58:27.946086 18460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca1ca0f378247598c915c11faee8ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:27.893261 18661 consensus_queue.cc:260] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
W20250112 20:58:27.973881 18342 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
W20250112 20:58:27.970984 18280 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.131:39861, user_credentials={real_user=slave}} blocked reactor thread for 68421.1us
I20250112 20:58:27.978608 18379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57802 (request call id 35) took 1199 ms. Trace:
I20250112 20:58:27.988976 18379 rpcz_store.cc:276] 0112 20:58:26.778937 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.414273 (+635336us) service_pool.cc:225] Handling call
0112 20:58:27.505102 (+ 90829us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.505226 (+ 124us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.781961 (+276735us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:27.974402 (+192441us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:27.978547 (+ 4145us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20250112 20:58:27.990573 18281 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 87438.1us
I20250112 20:58:28.006469 18532 tablet_service.cc:1467] Processing CreateTablet for tablet f8820dfc60bc477985f861406c22dd3b (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20250112 20:58:28.007741 18532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8820dfc60bc477985f861406c22dd3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.015718 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c257b2d0772f4078b2ca2d7c1298e4ae" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:28.017140 18584 leader_election.cc:343] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.015903 18456 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 29) took 1188 ms. Trace:
I20250112 20:58:28.033326 18453 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 33) took 1162 ms. Trace:
I20250112 20:58:28.033710 18534 tablet_service.cc:1467] Processing CreateTablet for tablet 985f9b21969e48739677e3cdc86e25de (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20250112 20:58:28.037940 18377 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57802 (request call id 30) took 1275 ms. Trace:
I20250112 20:58:28.038350 18377 rpcz_store.cc:276] 0112 20:58:26.762936 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.273623 (+510687us) service_pool.cc:225] Handling call
0112 20:58:27.329970 (+ 56347us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.330057 (+ 87us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.796299 (+466242us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.037452 (+241153us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.037880 (+ 428us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.042994 18661 ts_tablet_manager.cc:1428] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.190s user 0.007s sys 0.000s
I20250112 20:58:28.044303 18661 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:28.044790 18664 tablet_bootstrap.cc:492] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.045269 18664 ts_tablet_manager.cc:1397] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.847s user 0.012s sys 0.041s
I20250112 20:58:28.019201 18456 rpcz_store.cc:276] 0112 20:58:26.827720 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.526154 (+698434us) service_pool.cc:225] Handling call
0112 20:58:27.528301 (+ 2147us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.528395 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.680282 (+151887us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.015387 (+335105us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.015850 (+ 463us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.050364 18456 tablet_service.cc:1467] Processing CreateTablet for tablet e8704b5088604084a641aa00baf84be8 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250112 20:58:28.052569 18533 tablet_service.cc:1467] Processing CreateTablet for tablet 161a73359f3a4bb19a4df8ad30c6669e (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20250112 20:58:28.053500 18604 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43562 (request call id 31) took 1136 ms. Trace:
I20250112 20:58:28.053889 18457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db2806263c3471cac0496b194a73ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.053952 18604 rpcz_store.cc:276] 0112 20:58:26.917348 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.498561 (+581213us) service_pool.cc:225] Handling call
0112 20:58:27.587021 (+ 88460us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.587112 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.802158 (+215046us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.052902 (+250744us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.053442 (+ 540us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.057524 18529 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 21) took 1022 ms. Trace:
I20250112 20:58:28.057910 18529 rpcz_store.cc:276] 0112 20:58:27.054815 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.458706 (+403891us) service_pool.cc:225] Handling call
0112 20:58:27.582072 (+123366us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.582213 (+ 141us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.881584 (+299371us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.057042 (+175458us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.057456 (+ 414us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.060257 18529 tablet_service.cc:1467] Processing CreateTablet for tablet 08844b71d8c54556af7a5d625ed646c2 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20250112 20:58:27.970880 18663 consensus_queue.cc:260] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.048043 18664 raft_consensus.cc:357] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.062808 18664 raft_consensus.cc:383] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.063113 18664 raft_consensus.cc:738] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.066229 18605 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43562 (request call id 33) took 1066 ms. Trace:
I20250112 20:58:28.067816 18605 rpcz_store.cc:276] 0112 20:58:26.999890 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.528818 (+528928us) service_pool.cc:225] Handling call
0112 20:58:27.538365 (+ 9547us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.538472 (+ 107us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.028940 (+490468us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.065792 (+ 36852us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.066174 (+ 382us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.067446 18535 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 25) took 1011 ms. Trace:
I20250112 20:58:28.068753 18535 rpcz_store.cc:276] 0112 20:58:27.056078 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.745701 (+689623us) service_pool.cc:225] Handling call
0112 20:58:27.747358 (+ 1657us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.747463 (+ 105us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.874487 (+127024us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.066995 (+192508us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.067392 (+ 397us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.069458 18664 consensus_queue.cc:260] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.070753 18535 tablet_service.cc:1467] Processing CreateTablet for tablet 3f52ac1efd0e417393e5127713a43a18 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20250112 20:58:28.072062 18608 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43562 (request call id 29) took 1183 ms. Trace:
I20250112 20:58:28.072088 18535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f52ac1efd0e417393e5127713a43a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.073073 18455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 219a31736df4499fb96cacfe05747d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.074281 18663 ts_tablet_manager.cc:1428] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.281s user 0.009s sys 0.008s
I20250112 20:58:28.075240 18456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8704b5088604084a641aa00baf84be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.082758 18454 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 34) took 1211 ms. Trace:
I20250112 20:58:28.085605 18662 tablet_bootstrap.cc:492] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.088973 18662 ts_tablet_manager.cc:1397] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.501s user 0.056s sys 0.038s
I20250112 20:58:28.072474 18608 rpcz_store.cc:276] 0112 20:58:26.889119 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.494165 (+605046us) service_pool.cc:225] Handling call
0112 20:58:27.495778 (+ 1613us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.495863 (+ 85us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.826526 (+330663us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.071623 (+245097us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.072005 (+ 382us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.090282 18534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 985f9b21969e48739677e3cdc86e25de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.092717 18662 raft_consensus.cc:357] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.092943 18526 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 27) took 1036 ms. Trace:
I20250112 20:58:28.093463 18662 raft_consensus.cc:383] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.088793 18454 rpcz_store.cc:276] 0112 20:58:27.008812 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.753514 (+744702us) service_pool.cc:225] Handling call
0112 20:58:27.759113 (+ 5599us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.759195 (+ 82us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.058906 (+299711us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.082288 (+ 23382us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.082702 (+ 414us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.093878 18662 raft_consensus.cc:738] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.094436 18527 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 23) took 1039 ms. Trace:
I20250112 20:58:28.094729 18452 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 32) took 1266 ms. Trace:
I20250112 20:58:28.094807 18527 rpcz_store.cc:276] 0112 20:58:27.055488 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.633424 (+577936us) service_pool.cc:225] Handling call
0112 20:58:27.654615 (+ 21191us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.654757 (+ 142us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.989701 (+334944us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.093974 (+104273us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.094380 (+ 406us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.092967 18533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161a73359f3a4bb19a4df8ad30c6669e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.097234 18662 consensus_queue.cc:260] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.100962 18663 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:28.033775 18453 rpcz_store.cc:276] 0112 20:58:26.871219 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.715754 (+844535us) service_pool.cc:225] Handling call
0112 20:58:27.717350 (+ 1596us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.717446 (+ 96us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.023195 (+305749us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.032834 (+ 9639us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.033266 (+ 432us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.104229 18454 tablet_service.cc:1467] Processing CreateTablet for tablet cdbd6c5512664ac7a36a13bd1a89d75f (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250112 20:58:28.093544 18526 rpcz_store.cc:276] 0112 20:58:27.056700 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.877907 (+821207us) service_pool.cc:225] Handling call
0112 20:58:27.880868 (+ 2961us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.880969 (+ 101us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.002680 (+121711us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.092414 (+ 89734us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.092875 (+ 461us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.101969 18601 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43562 (request call id 27) took 1242 ms. Trace:
I20250112 20:58:28.098488 18461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afe68bc3c3a54ebb816b85651cfd9d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.124789 18601 rpcz_store.cc:276] 0112 20:58:26.859175 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.487383 (+628208us) service_pool.cc:225] Handling call
0112 20:58:27.519061 (+ 31678us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.519199 (+ 138us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.899226 (+380027us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.040180 (+140954us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.078390 (+ 38210us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.125939 18526 tablet_service.cc:1467] Processing CreateTablet for tablet d7daf053c1144c6797f1ac89fedd0fd0 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20250112 20:58:28.128232 18531 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 26) took 1071 ms. Trace:
I20250112 20:58:28.128584 18531 rpcz_store.cc:276] 0112 20:58:27.056383 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.826223 (+769840us) service_pool.cc:225] Handling call
0112 20:58:27.849306 (+ 23083us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.849394 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.087247 (+237853us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.127705 (+ 40458us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.128176 (+ 471us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.130497 18531 tablet_service.cc:1467] Processing CreateTablet for tablet e8704b5088604084a641aa00baf84be8 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20250112 20:58:28.110426 18453 tablet_service.cc:1467] Processing CreateTablet for tablet 6961f63ec790480aaa86f70431ed86c6 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20250112 20:58:28.095328 18452 rpcz_store.cc:276] 0112 20:58:26.870797 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.714820 (+844023us) service_pool.cc:225] Handling call
0112 20:58:27.716961 (+ 2141us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.717043 (+ 82us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.014256 (+297213us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.088429 (+ 74173us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.094675 (+ 6246us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.115634 18529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08844b71d8c54556af7a5d625ed646c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.118669 18527 tablet_service.cc:1467] Processing CreateTablet for tablet 6db2806263c3471cac0496b194a73ab9 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20250112 20:58:28.137311 18527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db2806263c3471cac0496b194a73ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.137732 18381 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57802 (request call id 37) took 1140 ms. Trace:
I20250112 20:58:28.138244 18526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7daf053c1144c6797f1ac89fedd0fd0. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:58:28.143734 18280 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.23.131:39861, user_credentials={real_user=slave}} blocked reactor thread for 47052.9us
I20250112 20:58:28.144755 18670 raft_consensus.cc:491] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.145219 18670 raft_consensus.cc:513] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.146481 18661 tablet_bootstrap.cc:654] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.147070 18670 leader_election.cc:290] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:28.148856 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ee370770b54ae593e07378574861d1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:28.149587 18454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdbd6c5512664ac7a36a13bd1a89d75f. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:58:28.150152 18361 leader_election.cc:343] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.150983 18664 ts_tablet_manager.cc:1428] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.105s user 0.006s sys 0.002s
I20250112 20:58:28.152051 18664 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:28.155731 18453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6961f63ec790480aaa86f70431ed86c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.156898 18459 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 30) took 1328 ms. Trace:
I20250112 20:58:28.157286 18459 rpcz_store.cc:276] 0112 20:58:26.828013 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.555204 (+727191us) service_pool.cc:225] Handling call
0112 20:58:27.557505 (+ 2301us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.557620 (+ 115us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.990918 (+433298us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.092139 (+101221us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.156844 (+ 64705us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.159016 18459 tablet_service.cc:1467] Processing CreateTablet for tablet f4bdf5e8ae124786b98df6013ed28460 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250112 20:58:28.166613 18663 tablet_bootstrap.cc:654] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.166867 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5547898f3d53475fb47696f4ddd6ef1c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.168309 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c257b2d0772f4078b2ca2d7c1298e4ae" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:28.138113 18381 rpcz_store.cc:276] 0112 20:58:26.997290 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.502911 (+505621us) service_pool.cc:225] Handling call
0112 20:58:27.504587 (+ 1676us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.504709 (+ 122us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:27.751907 (+247198us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.137229 (+385322us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.137681 (+ 452us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.171645 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ee370770b54ae593e07378574861d1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:28.173501 18360 leader_election.cc:343] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:28.173666 18435 leader_election.cc:343] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.173914 18360 leader_election.cc:304] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:28.174170 18435 leader_election.cc:304] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:28.175007 18670 raft_consensus.cc:2743] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:58:28.175562 18585 leader_election.cc:343] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.175784 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d939684f7a845f7b267cf420779b603" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.176009 18585 leader_election.cc:304] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:28.176908 18459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdf5e8ae124786b98df6013ed28460. 1 dirs total, 0 dirs full, 0 dirs failed
W20250112 20:58:28.177084 18360 leader_election.cc:343] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.177587 18360 leader_election.cc:304] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:28.177779 18689 raft_consensus.cc:2743] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:28.178565 18670 raft_consensus.cc:2743] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:28.180872 18452 tablet_service.cc:1467] Processing CreateTablet for tablet e73763ba5b1a4f59901b9e8f5eb6ab78 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250112 20:58:28.182291 18452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73763ba5b1a4f59901b9e8f5eb6ab78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.183815 18456 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 38) took 1152 ms. Trace:
I20250112 20:58:28.185500 18609 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:43562 (request call id 43) took 1040 ms. Trace:
I20250112 20:58:28.186439 18531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8704b5088604084a641aa00baf84be8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.187093 18668 raft_consensus.cc:2743] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:28.189168 18528 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 28) took 1132 ms. Trace:
I20250112 20:58:28.188858 18688 raft_consensus.cc:491] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.189551 18528 rpcz_store.cc:276] 0112 20:58:27.056996 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.883129 (+826133us) service_pool.cc:225] Handling call
0112 20:58:27.886088 (+ 2959us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.886194 (+ 106us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.056990 (+170796us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.188727 (+131737us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.189117 (+ 390us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.191820 18530 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 29) took 1134 ms. Trace:
I20250112 20:58:28.189772 18609 rpcz_store.cc:276] 0112 20:58:27.145448 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.844317 (+698869us) service_pool.cc:225] Handling call
0112 20:58:27.934815 (+ 90498us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.934920 (+ 105us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.147139 (+212219us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.185073 (+ 37934us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.185450 (+ 377us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.184348 18455 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 31) took 1356 ms. Trace:
I20250112 20:58:28.191589 18528 tablet_service.cc:1467] Processing CreateTablet for tablet cdbd6c5512664ac7a36a13bd1a89d75f (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20250112 20:58:28.192222 18530 rpcz_store.cc:276] 0112 20:58:27.057296 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.900516 (+843220us) service_pool.cc:225] Handling call
0112 20:58:27.906534 (+ 6018us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.906646 (+ 112us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.132243 (+225597us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.191359 (+ 59116us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.191765 (+ 406us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.200745 18533 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 32) took 1142 ms. Trace:
I20250112 20:58:28.200228 18668 raft_consensus.cc:491] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.186807 18457 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 35) took 1177 ms. Trace:
I20250112 20:58:28.202672 18457 rpcz_store.cc:276] 0112 20:58:27.009187 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.940635 (+931448us) service_pool.cc:225] Handling call
0112 20:58:27.975781 (+ 35146us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.981275 (+ 5494us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.060670 (+ 79395us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.186379 (+125709us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.186756 (+ 377us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.189679 18661 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.194347 18455 rpcz_store.cc:276] 0112 20:58:26.828323 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.704160 (+875837us) service_pool.cc:225] Handling call
0112 20:58:27.770217 (+ 66057us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.770303 (+ 86us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.083197 (+312894us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.183845 (+100648us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.184285 (+ 440us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.195818 18528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdbd6c5512664ac7a36a13bd1a89d75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.201498 18533 rpcz_store.cc:276] 0112 20:58:27.058203 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.051807 (+993604us) service_pool.cc:225] Handling call
0112 20:58:28.053416 (+ 1609us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.053500 (+ 84us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.180811 (+127311us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.200189 (+ 19378us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.200671 (+ 482us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.201644 18668 raft_consensus.cc:513] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.209966 18529 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 33) took 1151 ms. Trace:
I20250112 20:58:28.228425 18533 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20250112 20:58:28.206048 18458 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 28) took 1367 ms. Trace:
I20250112 20:58:28.205792 18380 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57802 (request call id 44) took 1059 ms. Trace:
I20250112 20:58:28.187187 18456 rpcz_store.cc:276] 0112 20:58:27.031271 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.049491 (+1018220us) service_pool.cc:225] Handling call
0112 20:58:28.050976 (+ 1485us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.051039 (+ 63us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.100824 (+ 49785us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.183365 (+ 82541us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.183760 (+ 395us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.229614 18380 rpcz_store.cc:276] 0112 20:58:27.146096 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.820988 (+674892us) service_pool.cc:225] Handling call
0112 20:58:27.923906 (+102918us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.924011 (+ 105us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.130114 (+206103us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.205333 (+ 75219us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.205735 (+ 402us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.209383 18535 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 34) took 1150 ms. Trace:
I20250112 20:58:28.230839 18535 rpcz_store.cc:276] 0112 20:58:27.058827 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.069931 (+1011104us) service_pool.cc:225] Handling call
0112 20:58:28.071658 (+ 1727us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.071743 (+ 85us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.157599 (+ 85856us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.208905 (+ 51306us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.209330 (+ 425us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.233147 18533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fc3d5bd7d04459c9c1e7ccb28eb2ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.234593 18535 tablet_service.cc:1467] Processing CreateTablet for tablet e73763ba5b1a4f59901b9e8f5eb6ab78 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20250112 20:58:28.235214 18454 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 39) took 1180 ms. Trace:
I20250112 20:58:28.229346 18458 rpcz_store.cc:276] 0112 20:58:26.827421 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.511326 (+683905us) service_pool.cc:225] Handling call
0112 20:58:27.643722 (+132396us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.663738 (+ 20016us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.047863 (+384125us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.193436 (+145573us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.194662 (+ 1226us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.235613 18454 rpcz_store.cc:276] 0112 20:58:27.054330 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.103200 (+1048870us) service_pool.cc:225] Handling call
0112 20:58:28.124099 (+ 20899us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.124209 (+ 110us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.186447 (+ 62238us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.234753 (+ 48306us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.235156 (+ 403us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.194106 18688 raft_consensus.cc:513] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.228727 18529 rpcz_store.cc:276] 0112 20:58:27.058501 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.059457 (+1000956us) service_pool.cc:225] Handling call
0112 20:58:28.060989 (+ 1532us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.061076 (+ 87us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.169214 (+108138us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.209497 (+ 40283us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.209916 (+ 419us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.229523 18527 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 35) took 1170 ms. Trace:
I20250112 20:58:28.205968 18530 tablet_service.cc:1467] Processing CreateTablet for tablet 4bed5fc535ff4870870d2b8f20d637d7 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20250112 20:58:28.195802 18662 ts_tablet_manager.cc:1428] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.106s user 0.008s sys 0.000s
I20250112 20:58:28.209961 18383 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57802 (request call id 41) took 1067 ms. Trace:
I20250112 20:58:28.241864 18661 ts_tablet_manager.cc:1397] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.198s user 0.014s sys 0.004s
I20250112 20:58:28.247681 18535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73763ba5b1a4f59901b9e8f5eb6ab78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.204545 18457 tablet_service.cc:1467] Processing CreateTablet for tablet 95e82471468e430b877403ce304eac78 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20250112 20:58:28.246230 18532 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 31) took 1186 ms. Trace:
I20250112 20:58:28.259793 18457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95e82471468e430b877403ce304eac78. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.260046 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:28.261443 18585 leader_election.cc:343] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.262423 18529 tablet_service.cc:1467] Processing CreateTablet for tablet f4bdf5e8ae124786b98df6013ed28460 (DEFAULT_TABLE table=test-workload [id=6c5cb2d03bd845d99cf1b24b2d70cdbc]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20250112 20:58:28.266641 18661 raft_consensus.cc:357] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.267625 18661 raft_consensus.cc:383] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.269742 18661 raft_consensus.cc:738] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.257197 18668 leader_election.cc:290] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:28.271682 18461 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 36) took 1262 ms. Trace:
I20250112 20:58:28.272007 18661 consensus_queue.cc:260] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.258644 18532 rpcz_store.cc:276] 0112 20:58:27.057858 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.005644 (+947786us) service_pool.cc:225] Handling call
0112 20:58:28.007306 (+ 1662us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.007392 (+ 86us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.170191 (+162799us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.220359 (+ 50168us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.244322 (+ 23963us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.282264 18459 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 42) took 1176 ms. Trace:
I20250112 20:58:28.282711 18459 rpcz_store.cc:276] 0112 20:58:27.105781 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.158204 (+1052423us) service_pool.cc:225] Handling call
0112 20:58:28.159784 (+ 1580us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.159892 (+ 108us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.253015 (+ 93123us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.281785 (+ 28770us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.282205 (+ 420us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.245970 18383 rpcz_store.cc:276] 0112 20:58:27.142540 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.767706 (+625166us) service_pool.cc:225] Handling call
0112 20:58:27.769452 (+ 1746us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.769551 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.006285 (+236734us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.209398 (+203113us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.209892 (+ 494us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.273931 18664 tablet_bootstrap.cc:654] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.297256 18529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdf5e8ae124786b98df6013ed28460. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.274109 18526 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 37) took 1214 ms. Trace:
I20250112 20:58:28.285583 18688 leader_election.cc:290] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:28.240015 18527 rpcz_store.cc:276] 0112 20:58:27.059125 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.096364 (+1037239us) service_pool.cc:225] Handling call
0112 20:58:28.136884 (+ 40520us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.136975 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.159995 (+ 23020us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.228981 (+ 68986us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.229468 (+ 487us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.299705 18661 ts_tablet_manager.cc:1428] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.052s user 0.007s sys 0.000s
I20250112 20:58:28.299849 18526 rpcz_store.cc:276] 0112 20:58:27.059720 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.124921 (+1065201us) service_pool.cc:225] Handling call
0112 20:58:28.126792 (+ 1871us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.126875 (+ 83us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.211102 (+ 84227us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.273584 (+ 62482us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.274052 (+ 468us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.293936 18534 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 30) took 1236 ms. Trace:
I20250112 20:58:28.301934 18661 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:28.298606 18663 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.291603 18453 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 40) took 1237 ms. Trace:
I20250112 20:58:28.259141 18662 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:28.301852 18534 rpcz_store.cc:276] 0112 20:58:27.057579 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.990846 (+933267us) service_pool.cc:225] Handling call
0112 20:58:28.034885 (+ 44039us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.035018 (+ 133us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.204748 (+169730us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.293497 (+ 88749us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.293884 (+ 387us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.272094 18461 rpcz_store.cc:276] 0112 20:58:27.009647 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.873099 (+863452us) service_pool.cc:225] Handling call
0112 20:58:27.902229 (+ 29130us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.902328 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.203746 (+301418us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.271188 (+ 67442us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.271627 (+ 439us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.309373 18460 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 37) took 1278 ms. Trace:
I20250112 20:58:28.313230 18452 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 41) took 1217 ms. Trace:
I20250112 20:58:28.306582 18453 rpcz_store.cc:276] 0112 20:58:27.054632 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.109313 (+1054681us) service_pool.cc:225] Handling call
0112 20:58:28.132550 (+ 23237us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.155284 (+ 22734us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.242983 (+ 87699us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.291074 (+ 48091us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.291545 (+ 471us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.317286 18663 ts_tablet_manager.cc:1397] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.217s user 0.011s sys 0.010s
I20250112 20:58:28.320272 18530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bed5fc535ff4870870d2b8f20d637d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:58:28.318137 18460 rpcz_store.cc:276] 0112 20:58:27.030860 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:27.776563 (+745703us) service_pool.cc:225] Handling call
0112 20:58:27.780474 (+ 3911us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:27.780568 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.210893 (+430325us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.308854 (+ 97961us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.309318 (+ 464us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.325886 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
W20250112 20:58:28.330873 18583 leader_election.cc:343] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.331326 18583 leader_election.cc:304] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:28.332476 18668 raft_consensus.cc:2743] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:28.329322 18533 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 43) took 1211 ms. Trace:
I20250112 20:58:28.325119 18663 raft_consensus.cc:357] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.323710 18528 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 39) took 1207 ms. Trace:
I20250112 20:58:28.322669 18452 rpcz_store.cc:276] 0112 20:58:27.095875 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.133649 (+1037774us) service_pool.cc:225] Handling call
0112 20:58:28.181840 (+ 48191us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.181936 (+ 96us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.199930 (+ 17994us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.312803 (+112873us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.313177 (+ 374us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.335242 18528 rpcz_store.cc:276] 0112 20:58:27.116723 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.190695 (+1073972us) service_pool.cc:225] Handling call
0112 20:58:28.195373 (+ 4678us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.195470 (+ 97us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.270394 (+ 74924us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.323237 (+ 52843us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.323655 (+ 418us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.334805 18663 raft_consensus.cc:383] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.327510 18531 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 38) took 1236 ms. Trace:
I20250112 20:58:28.336251 18663 raft_consensus.cc:738] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.336653 18531 rpcz_store.cc:276] 0112 20:58:27.090881 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.129719 (+1038838us) service_pool.cc:225] Handling call
0112 20:58:28.131218 (+ 1499us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.131300 (+ 82us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.303318 (+172018us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.327064 (+ 23746us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.327458 (+ 394us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.320941 18662 tablet_bootstrap.cc:654] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.337182 18663 consensus_queue.cc:260] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.333400 18533 rpcz_store.cc:276] 0112 20:58:27.118358 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.227380 (+1109022us) service_pool.cc:225] Handling call
0112 20:58:28.232739 (+ 5359us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.232827 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.267401 (+ 34574us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.328866 (+ 61465us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.329261 (+ 395us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.341650 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1906b96af18e4a7184907ae1b525aa02" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:28.342248 18471 raft_consensus.cc:2463] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:28.343510 18511 leader_election.cc:304] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:28.346239 18688 raft_consensus.cc:2798] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.347159 18457 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:48902 (request call id 43) took 1232 ms. Trace:
I20250112 20:58:28.347391 18688 raft_consensus.cc:491] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.347553 18457 rpcz_store.cc:276] 0112 20:58:27.115204 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.203618 (+1088414us) service_pool.cc:225] Handling call
0112 20:58:28.259134 (+ 55516us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.259297 (+ 163us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.313944 (+ 54647us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.346698 (+ 32754us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.347108 (+ 410us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.347960 18688 raft_consensus.cc:3054] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.349262 18663 ts_tablet_manager.cc:1428] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.028s user 0.004s sys 0.003s
I20250112 20:58:28.349882 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1906b96af18e4a7184907ae1b525aa02" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.350703 18663 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
W20250112 20:58:28.351147 18509 leader_election.cc:343] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.359095 18661 tablet_bootstrap.cc:654] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.362660 18662 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.363106 18662 ts_tablet_manager.cc:1397] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.104s user 0.012s sys 0.008s
I20250112 20:58:28.363281 18664 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.363673 18664 ts_tablet_manager.cc:1397] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.212s user 0.017s sys 0.011s
I20250112 20:58:28.365682 18662 raft_consensus.cc:357] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.365962 18664 raft_consensus.cc:357] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.366425 18662 raft_consensus.cc:383] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.366714 18664 raft_consensus.cc:383] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.366798 18662 raft_consensus.cc:738] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.367228 18661 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.367846 18661 ts_tablet_manager.cc:1397] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.066s user 0.008s sys 0.004s
I20250112 20:58:28.367702 18662 consensus_queue.cc:260] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.367111 18664 raft_consensus.cc:738] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.369015 18664 consensus_queue.cc:260] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.372241 18688 raft_consensus.cc:513] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.370373 18662 ts_tablet_manager.cc:1428] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250112 20:58:28.374965 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1906b96af18e4a7184907ae1b525aa02" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:28.375542 18471 raft_consensus.cc:3054] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.375861 18662 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:28.371781 18661 raft_consensus.cc:357] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.381614 18661 raft_consensus.cc:383] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.381922 18661 raft_consensus.cc:738] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.382637 18661 consensus_queue.cc:260] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.384959 18661 ts_tablet_manager.cc:1428] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20250112 20:58:28.385360 18663 tablet_bootstrap.cc:654] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.386610 18471 raft_consensus.cc:2463] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:28.387701 18511 leader_election.cc:304] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:28.393467 18535 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 41) took 1276 ms. Trace:
I20250112 20:58:28.394006 18535 rpcz_store.cc:276] 0112 20:58:27.117527 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.233763 (+1116236us) service_pool.cc:225] Handling call
0112 20:58:28.235436 (+ 1673us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.235527 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.359599 (+124072us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.392761 (+ 33162us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.393391 (+ 630us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.396548 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1906b96af18e4a7184907ae1b525aa02" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
W20250112 20:58:28.397794 18509 leader_election.cc:343] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.398277 18661 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:28.398646 18688 leader_election.cc:290] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:28.399374 18688 raft_consensus.cc:2798] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.399794 18688 raft_consensus.cc:695] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:28.400565 18688 consensus_queue.cc:237] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.405772 18662 tablet_bootstrap.cc:654] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.405843 18661 tablet_bootstrap.cc:654] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.410903 18664 ts_tablet_manager.cc:1428] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.047s user 0.031s sys 0.012s
I20250112 20:58:28.411940 18664 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:28.414732 18661 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.415308 18661 ts_tablet_manager.cc:1397] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.000s
I20250112 20:58:28.417712 18663 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.418002 18664 tablet_bootstrap.cc:654] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.418200 18663 ts_tablet_manager.cc:1397] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.068s user 0.009s sys 0.003s
I20250112 20:58:28.418805 18661 raft_consensus.cc:357] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.419751 18661 raft_consensus.cc:383] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.420097 18661 raft_consensus.cc:738] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.420900 18663 raft_consensus.cc:357] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.421974 18663 raft_consensus.cc:383] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.422363 18663 raft_consensus.cc:738] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.423738 18530 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 40) took 1306 ms. Trace:
I20250112 20:58:28.423944 18661 consensus_queue.cc:260] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.427747 18662 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.428270 18662 ts_tablet_manager.cc:1397] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.053s user 0.010s sys 0.005s
I20250112 20:58:28.427588 18290 catalog_manager.cc:5526] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:28.429947 18664 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.430408 18664 ts_tablet_manager.cc:1397] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20250112 20:58:28.430814 18662 raft_consensus.cc:357] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.431254 18529 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46080 (request call id 42) took 1313 ms. Trace:
I20250112 20:58:28.431600 18662 raft_consensus.cc:383] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.432026 18662 raft_consensus.cc:738] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.424266 18530 rpcz_store.cc:276] 0112 20:58:27.117143 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.205164 (+1088021us) service_pool.cc:225] Handling call
0112 20:58:28.245727 (+ 40563us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.253218 (+ 7491us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.357736 (+104518us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.415405 (+ 57669us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.423669 (+ 8264us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.432904 18662 consensus_queue.cc:260] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.431691 18529 rpcz_store.cc:276] 0112 20:58:27.117905 (+ 0us) service_pool.cc:168] Inserting onto call queue
0112 20:58:28.244788 (+1126883us) service_pool.cc:225] Handling call
0112 20:58:28.263159 (+ 18371us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0112 20:58:28.263241 (+ 82us) ts_tablet_manager.cc:676] Creating new metadata...
0112 20:58:28.373671 (+110430us) tablet_metadata.cc:641] Metadata flushed
0112 20:58:28.423158 (+ 49487us) tablet_replica.cc:206] Creating consensus instance
0112 20:58:28.431173 (+ 8015us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250112 20:58:28.435125 18661 ts_tablet_manager.cc:1428] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20250112 20:58:28.435106 18663 consensus_queue.cc:260] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.437805 18662 ts_tablet_manager.cc:1428] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20250112 20:58:28.437830 18663 ts_tablet_manager.cc:1428] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.019s user 0.004s sys 0.004s
I20250112 20:58:28.438743 18662 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:28.439046 18663 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:28.433372 18664 raft_consensus.cc:357] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.439738 18664 raft_consensus.cc:383] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.440033 18664 raft_consensus.cc:738] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.440798 18664 consensus_queue.cc:260] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.441902 18661 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:28.443032 18664 ts_tablet_manager.cc:1428] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.012s user 0.003s sys 0.001s
I20250112 20:58:28.444005 18664 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:28.445293 18662 tablet_bootstrap.cc:654] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.445827 18663 tablet_bootstrap.cc:654] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.455909 18664 tablet_bootstrap.cc:654] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.456753 18662 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.457268 18662 ts_tablet_manager.cc:1397] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.000s
I20250112 20:58:28.460075 18661 tablet_bootstrap.cc:654] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.461836 18662 raft_consensus.cc:357] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.462756 18662 raft_consensus.cc:383] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.463112 18662 raft_consensus.cc:738] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.464027 18662 consensus_queue.cc:260] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.467310 18662 ts_tablet_manager.cc:1428] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250112 20:58:28.468175 18662 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:28.476437 18664 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.477903 18664 ts_tablet_manager.cc:1397] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.010s
I20250112 20:58:28.479622 18662 tablet_bootstrap.cc:654] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.482331 18661 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.482038 18664 raft_consensus.cc:357] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.482712 18661 ts_tablet_manager.cc:1397] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.004s
I20250112 20:58:28.482793 18664 raft_consensus.cc:383] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.483136 18664 raft_consensus.cc:738] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.483235 18663 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.483743 18663 ts_tablet_manager.cc:1397] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.045s user 0.015s sys 0.009s
I20250112 20:58:28.485272 18661 raft_consensus.cc:357] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.486042 18661 raft_consensus.cc:383] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.486387 18661 raft_consensus.cc:738] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.486891 18663 raft_consensus.cc:357] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.487051 18661 consensus_queue.cc:260] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.487841 18663 raft_consensus.cc:383] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.488220 18663 raft_consensus.cc:738] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.488862 18663 consensus_queue.cc:260] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.491168 18663 ts_tablet_manager.cc:1428] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250112 20:58:28.493283 18661 ts_tablet_manager.cc:1428] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20250112 20:58:28.492152 18663 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:28.492743 18664 consensus_queue.cc:260] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.498634 18664 ts_tablet_manager.cc:1428] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20250112 20:58:28.499467 18664 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:28.503496 18663 tablet_bootstrap.cc:654] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.506248 18664 tablet_bootstrap.cc:654] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.506903 18661 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:28.507937 18689 raft_consensus.cc:491] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.508410 18689 raft_consensus.cc:513] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.509229 18662 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.509827 18662 ts_tablet_manager.cc:1397] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.042s user 0.012s sys 0.009s
I20250112 20:58:28.510622 18689 leader_election.cc:290] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:28.511878 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4295797580d643d88c1ef2701a123bef" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:28.512735 18545 raft_consensus.cc:2463] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:28.515738 18436 leader_election.cc:304] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:28.516391 18661 tablet_bootstrap.cc:654] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.516734 18689 raft_consensus.cc:2798] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.517127 18689 raft_consensus.cc:491] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.517434 18689 raft_consensus.cc:3054] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.519003 18662 raft_consensus.cc:357] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.519769 18662 raft_consensus.cc:383] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.520109 18662 raft_consensus.cc:738] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.520846 18662 consensus_queue.cc:260] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.524029 18664 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.524227 18689 raft_consensus.cc:513] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.526630 18664 ts_tablet_manager.cc:1397] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.009s
I20250112 20:58:28.528447 18661 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.528914 18661 ts_tablet_manager.cc:1397] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.007s
I20250112 20:58:28.529306 18663 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.529778 18663 ts_tablet_manager.cc:1397] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.038s user 0.010s sys 0.001s
I20250112 20:58:28.531029 18664 raft_consensus.cc:357] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.531332 18661 raft_consensus.cc:357] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.531773 18664 raft_consensus.cc:383] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.531994 18661 raft_consensus.cc:383] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.532073 18664 raft_consensus.cc:738] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.532352 18661 raft_consensus.cc:738] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.532514 18663 raft_consensus.cc:357] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.532800 18664 consensus_queue.cc:260] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.533332 18663 raft_consensus.cc:383] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.533192 18661 consensus_queue.cc:260] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.533751 18663 raft_consensus.cc:738] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.534694 18689 leader_election.cc:290] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:28.535713 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4295797580d643d88c1ef2701a123bef" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:28.536252 18545 raft_consensus.cc:3054] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.536451 18670 raft_consensus.cc:491] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.536888 18670 raft_consensus.cc:513] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.538393 18661 ts_tablet_manager.cc:1428] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20250112 20:58:28.538913 18670 leader_election.cc:290] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:28.539294 18661 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:28.541963 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59cd634f5244017b18a111892243f57" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.542629 18619 raft_consensus.cc:2463] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:28.535553 18663 consensus_queue.cc:260] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.543740 18664 ts_tablet_manager.cc:1428] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20250112 20:58:28.544565 18360 leader_election.cc:304] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.550387 18663 ts_tablet_manager.cc:1428] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.020s user 0.005s sys 0.002s
I20250112 20:58:28.550637 18664 tablet_bootstrap.cc:492] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:28.550894 18670 raft_consensus.cc:2798] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.551249 18670 raft_consensus.cc:491] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.551563 18670 raft_consensus.cc:3054] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.552515 18663 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:28.556073 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59cd634f5244017b18a111892243f57" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:28.556385 18661 tablet_bootstrap.cc:654] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.556779 18471 raft_consensus.cc:2463] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:28.564261 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4295797580d643d88c1ef2701a123bef" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:28.564898 18396 raft_consensus.cc:2463] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:28.565944 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4295797580d643d88c1ef2701a123bef" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:28.566565 18395 raft_consensus.cc:3054] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.566617 18663 tablet_bootstrap.cc:654] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.570202 18670 raft_consensus.cc:513] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.571321 18664 tablet_bootstrap.cc:654] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.572836 18670 leader_election.cc:290] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:28.571192 18545 raft_consensus.cc:2463] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:28.574384 18436 leader_election.cc:304] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:28.575134 18689 raft_consensus.cc:2798] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.575341 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59cd634f5244017b18a111892243f57" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:28.575587 18689 raft_consensus.cc:695] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:28.573166 18395 raft_consensus.cc:2463] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:28.576982 18619 raft_consensus.cc:3054] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.578469 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59cd634f5244017b18a111892243f57" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:28.578271 18689 consensus_queue.cc:237] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.579016 18471 raft_consensus.cc:3054] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.576761 18670 raft_consensus.cc:491] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.590000 18619 raft_consensus.cc:2463] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:28.590917 18662 ts_tablet_manager.cc:1428] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.081s user 0.004s sys 0.007s
I20250112 20:58:28.590979 18360 leader_election.cc:304] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.592072 18662 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:28.592012 18719 raft_consensus.cc:2798] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.592296 18471 raft_consensus.cc:2463] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:28.586408 18670 raft_consensus.cc:513] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.595136 18661 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.595585 18661 ts_tablet_manager.cc:1397] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.056s user 0.004s sys 0.010s
I20250112 20:58:28.596494 18663 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.597004 18663 ts_tablet_manager.cc:1397] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.046s user 0.012s sys 0.010s
I20250112 20:58:28.598115 18661 raft_consensus.cc:357] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.598776 18661 raft_consensus.cc:383] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.599020 18661 raft_consensus.cc:738] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.599041 18670 leader_election.cc:290] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:28.599717 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.599659 18661 consensus_queue.cc:260] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:28.600906 18360 leader_election.cc:343] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.602450 18663 raft_consensus.cc:357] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.604041 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:28.604573 18663 raft_consensus.cc:383] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.605150 18663 raft_consensus.cc:738] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
W20250112 20:58:28.605569 18362 leader_election.cc:343] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.605798 18689 raft_consensus.cc:491] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.606042 18362 leader_election.cc:304] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:28.606231 18689 raft_consensus.cc:513] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.606875 18663 consensus_queue.cc:260] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.608703 18670 raft_consensus.cc:2743] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:28.609162 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3629a889ed427db7d163963dd13507" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.609869 18619 raft_consensus.cc:2463] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:28.610177 18662 tablet_bootstrap.cc:654] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.610929 18435 leader_election.cc:304] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.611938 18718 raft_consensus.cc:2798] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.612408 18718 raft_consensus.cc:491] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.612839 18718 raft_consensus.cc:3054] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.613406 18661 ts_tablet_manager.cc:1428] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20250112 20:58:28.614285 18661 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:28.610703 18292 catalog_manager.cc:5526] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:28.620152 18719 raft_consensus.cc:695] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:28.621074 18719 consensus_queue.cc:237] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.617977 18663 ts_tablet_manager.cc:1428] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20250112 20:58:28.624253 18663 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:28.630693 18661 tablet_bootstrap.cc:654] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.633536 18663 tablet_bootstrap.cc:654] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.639382 18662 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.639919 18662 ts_tablet_manager.cc:1397] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.048s user 0.011s sys 0.000s
I20250112 20:58:28.643164 18662 raft_consensus.cc:357] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.644093 18662 raft_consensus.cc:383] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.644444 18718 raft_consensus.cc:513] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.645069 18689 leader_election.cc:290] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.646198 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3629a889ed427db7d163963dd13507" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:28.646595 18718 leader_election.cc:290] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.646844 18395 raft_consensus.cc:2463] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:28.648907 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3629a889ed427db7d163963dd13507" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:28.649484 18619 raft_consensus.cc:3054] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.644438 18662 raft_consensus.cc:738] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.650619 18664 tablet_bootstrap.cc:492] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.651095 18664 ts_tablet_manager.cc:1397] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.101s user 0.021s sys 0.037s
I20250112 20:58:28.651643 18661 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.650727 18662 consensus_queue.cc:260] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.652065 18661 ts_tablet_manager.cc:1397] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.000s
I20250112 20:58:28.653715 18662 ts_tablet_manager.cc:1428] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.013s user 0.007s sys 0.001s
I20250112 20:58:28.653723 18664 raft_consensus.cc:357] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.654209 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3629a889ed427db7d163963dd13507" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:28.654604 18664 raft_consensus.cc:383] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.654717 18662 tablet_bootstrap.cc:492] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:28.654937 18664 raft_consensus.cc:738] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.655764 18664 consensus_queue.cc:260] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.658911 18619 raft_consensus.cc:2463] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:28.654762 18395 raft_consensus.cc:3054] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.661000 18662 tablet_bootstrap.cc:654] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.663507 18668 raft_consensus.cc:491] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.664078 18668 raft_consensus.cc:513] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.664327 18661 raft_consensus.cc:357] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.665158 18661 raft_consensus.cc:383] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.665421 18661 raft_consensus.cc:738] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.665961 18661 consensus_queue.cc:260] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.666445 18668 leader_election.cc:290] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.667693 18664 ts_tablet_manager.cc:1428] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20250112 20:58:28.668305 18661 ts_tablet_manager.cc:1428] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20250112 20:58:28.669323 18661 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:28.668291 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d939684f7a845f7b267cf420779b603" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:28.670010 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d939684f7a845f7b267cf420779b603" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:28.670644 18396 raft_consensus.cc:2463] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
W20250112 20:58:28.671015 18584 leader_election.cc:343] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.671782 18662 tablet_bootstrap.cc:492] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.671679 18583 leader_election.cc:304] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:28.672237 18662 ts_tablet_manager.cc:1397] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20250112 20:58:28.672741 18668 raft_consensus.cc:2798] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.673113 18668 raft_consensus.cc:491] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.673445 18668 raft_consensus.cc:3054] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.674100 18435 leader_election.cc:304] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.674603 18664 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:28.674831 18718 raft_consensus.cc:2798] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.675369 18718 raft_consensus.cc:695] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:28.676262 18395 raft_consensus.cc:2463] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:28.676147 18718 consensus_queue.cc:237] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.679730 18663 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.680305 18663 ts_tablet_manager.cc:1397] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.056s user 0.012s sys 0.001s
I20250112 20:58:28.681651 18664 tablet_bootstrap.cc:654] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.674590 18662 raft_consensus.cc:357] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.682097 18662 raft_consensus.cc:383] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.682399 18662 raft_consensus.cc:738] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.682843 18719 raft_consensus.cc:491] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.682438 18294 catalog_manager.cc:5526] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:28.683390 18719 raft_consensus.cc:513] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.684473 18662 consensus_queue.cc:260] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.686151 18719 leader_election.cc:290] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:28.686103 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:28.686908 18471 raft_consensus.cc:2463] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:28.687847 18662 ts_tablet_manager.cc:1428] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20250112 20:58:28.687744 18362 leader_election.cc:304] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:28.688724 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:28.689028 18719 raft_consensus.cc:2798] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.689687 18719 raft_consensus.cc:491] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.690115 18719 raft_consensus.cc:3054] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.688768 18662 tablet_bootstrap.cc:492] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
W20250112 20:58:28.691525 18361 leader_election.cc:343] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.694892 18663 raft_consensus.cc:357] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.695638 18663 raft_consensus.cc:383] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.700363 18663 raft_consensus.cc:738] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.696434 18662 tablet_bootstrap.cc:654] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.701263 18663 consensus_queue.cc:260] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.699680 18719 raft_consensus.cc:513] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.705004 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:28.705583 18471 raft_consensus.cc:3054] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.705284 18719 leader_election.cc:290] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:28.707855 18663 ts_tablet_manager.cc:1428] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.027s user 0.005s sys 0.002s
I20250112 20:58:28.709267 18663 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:28.711172 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
W20250112 20:58:28.712428 18361 leader_election.cc:343] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.713146 18668 raft_consensus.cc:513] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.714788 18663 tablet_bootstrap.cc:654] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.715463 18471 raft_consensus.cc:2463] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:28.715628 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d939684f7a845f7b267cf420779b603" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:28.716511 18362 leader_election.cc:304] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
W20250112 20:58:28.716917 18584 leader_election.cc:343] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.717686 18719 raft_consensus.cc:2798] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.718016 18664 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.718477 18664 ts_tablet_manager.cc:1397] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.044s user 0.014s sys 0.006s
I20250112 20:58:28.718947 18719 raft_consensus.cc:695] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:28.720286 18661 tablet_bootstrap.cc:654] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.719908 18719 consensus_queue.cc:237] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.721098 18664 raft_consensus.cc:357] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.721868 18664 raft_consensus.cc:383] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.722165 18664 raft_consensus.cc:738] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.721555 18294 catalog_manager.cc:5526] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:28.722936 18664 consensus_queue.cc:260] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.723860 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d939684f7a845f7b267cf420779b603" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:28.724448 18395 raft_consensus.cc:3054] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.726981 18662 tablet_bootstrap.cc:492] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.727514 18662 ts_tablet_manager.cc:1397] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.002s
I20250112 20:58:28.730448 18662 raft_consensus.cc:357] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.731251 18662 raft_consensus.cc:383] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.731563 18662 raft_consensus.cc:738] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.732208 18664 ts_tablet_manager.cc:1428] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.013s user 0.004s sys 0.003s
I20250112 20:58:28.732483 18662 consensus_queue.cc:260] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.735422 18662 ts_tablet_manager.cc:1428] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250112 20:58:28.736443 18662 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:28.738664 18395 raft_consensus.cc:2463] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:28.738986 18688 raft_consensus.cc:491] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.739436 18688 raft_consensus.cc:513] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.739733 18583 leader_election.cc:304] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:28.740638 18668 leader_election.cc:290] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.740677 18732 raft_consensus.cc:2798] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.742100 18688 leader_election.cc:290] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:28.743574 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5547898f3d53475fb47696f4ddd6ef1c" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.743865 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5547898f3d53475fb47696f4ddd6ef1c" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:28.744411 18471 raft_consensus.cc:2463] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
W20250112 20:58:28.744879 18509 leader_election.cc:343] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.745494 18511 leader_election.cc:304] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:28.746708 18688 raft_consensus.cc:2798] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.747053 18688 raft_consensus.cc:491] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.747422 18688 raft_consensus.cc:3054] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.752702 18732 raft_consensus.cc:695] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:28.754703 18664 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:28.754202 18732 consensus_queue.cc:237] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.762545 18662 tablet_bootstrap.cc:654] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.767396 18661 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.767807 18661 ts_tablet_manager.cc:1397] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.099s user 0.018s sys 0.012s
I20250112 20:58:28.770502 18661 raft_consensus.cc:357] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.771206 18661 raft_consensus.cc:383] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.770776 18688 raft_consensus.cc:513] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.771505 18661 raft_consensus.cc:738] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.771991 18663 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.772500 18663 ts_tablet_manager.cc:1397] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.063s user 0.016s sys 0.012s
I20250112 20:58:28.772230 18661 consensus_queue.cc:260] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.773761 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5547898f3d53475fb47696f4ddd6ef1c" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:28.773919 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5547898f3d53475fb47696f4ddd6ef1c" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:28.774402 18661 ts_tablet_manager.cc:1428] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250112 20:58:28.775239 18661 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:28.775270 18509 leader_election.cc:343] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.775854 18688 leader_election.cc:290] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:28.775432 18663 raft_consensus.cc:357] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.777483 18663 raft_consensus.cc:383] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.777751 18663 raft_consensus.cc:738] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.778793 18663 consensus_queue.cc:260] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.780503 18661 tablet_bootstrap.cc:654] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.781558 18663 ts_tablet_manager.cc:1428] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250112 20:58:28.774374 18471 raft_consensus.cc:3054] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.782614 18663 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:28.787348 18290 catalog_manager.cc:5526] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 1 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:28.777740 18294 catalog_manager.cc:5526] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:28.788964 18471 raft_consensus.cc:2463] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:28.789420 18663 tablet_bootstrap.cc:654] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.790134 18511 leader_election.cc:304] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:28.791224 18688 raft_consensus.cc:2798] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.791698 18688 raft_consensus.cc:695] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:28.792490 18688 consensus_queue.cc:237] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.799365 18662 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:28.799856 18662 ts_tablet_manager.cc:1397] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.064s user 0.012s sys 0.004s
I20250112 20:58:28.806421 18662 raft_consensus.cc:357] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.807089 18662 raft_consensus.cc:383] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.807401 18662 raft_consensus.cc:738] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
W20250112 20:58:28.812911 18509 consensus_peers.cc:487] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb -> Peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Couldn't send request to peer c60c0cbe377c450297e3d536b7f7c89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:28.814191 18664 tablet_bootstrap.cc:654] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.818881 18718 raft_consensus.cc:491] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.819362 18718 raft_consensus.cc:513] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.821885 18718 leader_election.cc:290] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:28.823698 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c257b2d0772f4078b2ca2d7c1298e4ae" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.824373 18619 raft_consensus.cc:2463] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:28.825523 18435 leader_election.cc:304] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.826889 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c257b2d0772f4078b2ca2d7c1298e4ae" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:28.827435 18738 log.cc:926] Time spent T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
W20250112 20:58:28.828292 18436 leader_election.cc:343] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.829289 18718 raft_consensus.cc:2798] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.829814 18718 raft_consensus.cc:491] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.830220 18718 raft_consensus.cc:3054] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.833636 18664 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.834079 18664 ts_tablet_manager.cc:1397] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.080s user 0.010s sys 0.004s
I20250112 20:58:28.834719 18292 catalog_manager.cc:5526] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:28.836359 18664 raft_consensus.cc:357] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.836982 18664 raft_consensus.cc:383] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.837255 18664 raft_consensus.cc:738] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.837989 18664 consensus_queue.cc:260] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.839378 18698 raft_consensus.cc:491] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.839830 18698 raft_consensus.cc:513] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.844658 18698 leader_election.cc:290] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:28.845078 18700 raft_consensus.cc:491] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.845330 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f492fad2f5422392d1d141e7534579" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:28.845666 18700 raft_consensus.cc:513] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.847244 18664 ts_tablet_manager.cc:1428] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20250112 20:58:28.848168 18700 leader_election.cc:290] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.858184 18663 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.848186 18664 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:28.852202 18718 raft_consensus.cc:513] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.854499 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ee370770b54ae593e07378574861d1" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.858685 18663 ts_tablet_manager.cc:1397] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.076s user 0.039s sys 0.031s
I20250112 20:58:28.859180 18619 raft_consensus.cc:2463] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:28.860119 18718 leader_election.cc:290] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:28.860358 18509 leader_election.cc:304] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.860822 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ee370770b54ae593e07378574861d1" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:28.807968 18662 consensus_queue.cc:260] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.861459 18396 raft_consensus.cc:2463] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:28.861372 18663 raft_consensus.cc:357] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.862169 18663 raft_consensus.cc:383] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.862488 18663 raft_consensus.cc:738] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.862371 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c257b2d0772f4078b2ca2d7c1298e4ae" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:28.863354 18662 ts_tablet_manager.cc:1428] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.063s user 0.006s sys 0.001s
I20250112 20:58:28.863145 18663 consensus_queue.cc:260] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:28.863790 18436 leader_election.cc:343] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.864672 18662 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:28.865095 18471 raft_consensus.cc:2463] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:28.865415 18663 ts_tablet_manager.cc:1428] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250112 20:58:28.866730 18663 tablet_bootstrap.cc:492] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:28.867966 18511 leader_election.cc:304] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:28.868849 18688 raft_consensus.cc:2798] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.869237 18688 raft_consensus.cc:491] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.869576 18688 raft_consensus.cc:3054] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.869807 18732 raft_consensus.cc:491] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.870409 18661 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:28.872330 18661 ts_tablet_manager.cc:1397] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.097s user 0.018s sys 0.049s
I20250112 20:58:28.870266 18732 raft_consensus.cc:513] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.872758 18698 raft_consensus.cc:2798] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.873067 18698 raft_consensus.cc:491] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.873324 18698 raft_consensus.cc:3054] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.873981 18732 leader_election.cc:290] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.874787 18661 raft_consensus.cc:357] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.875526 18661 raft_consensus.cc:383] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:28.860044 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f492fad2f5422392d1d141e7534579" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:28.871842 18663 tablet_bootstrap.cc:654] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.876132 18395 raft_consensus.cc:2463] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:28.877046 18662 tablet_bootstrap.cc:654] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.877383 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaf48f6743cb46e686eaa798a4a22957" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:28.875792 18661 raft_consensus.cc:738] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.864527 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c257b2d0772f4078b2ca2d7c1298e4ae" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:28.878638 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaf48f6743cb46e686eaa798a4a22957" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:28.878844 18619 raft_consensus.cc:3054] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.879243 18396 raft_consensus.cc:2463] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:28.880741 18583 leader_election.cc:304] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.878046 18545 raft_consensus.cc:2463] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:28.888576 18732 raft_consensus.cc:2798] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:28.889065 18732 raft_consensus.cc:491] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:28.889492 18732 raft_consensus.cc:3054] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.896744 18700 raft_consensus.cc:491] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.897292 18700 raft_consensus.cc:513] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.897295 18746 consensus_queue.cc:1035] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:28.899474 18700 leader_election.cc:290] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:28.901863 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:28.902864 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:28.903652 18470 raft_consensus.cc:2388] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 1: Already voted for candidate a45f68dd3d9043809f3bb06287f5d2d3 in this term.
I20250112 20:58:28.904760 18511 leader_election.cc:304] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:28.908998 18698 raft_consensus.cc:513] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.911978 18732 raft_consensus.cc:513] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.914016 18700 raft_consensus.cc:3054] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.914970 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaf48f6743cb46e686eaa798a4a22957" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:28.915553 18545 raft_consensus.cc:3054] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.913585 18688 raft_consensus.cc:513] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.917896 18736 raft_consensus.cc:491] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.918365 18736 raft_consensus.cc:513] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.919509 18688 leader_election.cc:290] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.920135 18688 raft_consensus.cc:491] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.920665 18688 raft_consensus.cc:513] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.922705 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ee370770b54ae593e07378574861d1" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:28.923401 18618 raft_consensus.cc:3054] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.929138 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ee370770b54ae593e07378574861d1" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:28.925379 18700 raft_consensus.cc:2743] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:28.929867 18396 raft_consensus.cc:3054] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.931605 18619 raft_consensus.cc:2463] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:28.932987 18435 leader_election.cc:304] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:28.934356 18718 raft_consensus.cc:2798] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.934917 18718 raft_consensus.cc:695] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:28.935932 18718 consensus_queue.cc:237] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:28.938022 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:28.940232 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
W20250112 20:58:28.942080 18584 leader_election.cc:343] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:28.942911 18583 leader_election.cc:343] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:28.943432 18583 leader_election.cc:304] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:28.943572 18698 leader_election.cc:290] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:28.944206 18668 raft_consensus.cc:2743] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:28.946010 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f492fad2f5422392d1d141e7534579" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:28.946569 18395 raft_consensus.cc:3054] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.948002 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2f492fad2f5422392d1d141e7534579" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:28.948650 18736 leader_election.cc:290] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.948666 18471 raft_consensus.cc:3054] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.962855 18664 tablet_bootstrap.cc:654] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:28.965214 18618 raft_consensus.cc:2463] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:28.966300 18509 leader_election.cc:304] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.966980 18688 leader_election.cc:290] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.967684 18688 raft_consensus.cc:2798] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.968163 18471 raft_consensus.cc:2463] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:28.968443 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d939684f7a845f7b267cf420779b603" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:28.985725 18394 raft_consensus.cc:2388] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 1: Already voted for candidate c60c0cbe377c450297e3d536b7f7c89c in this term.
I20250112 20:58:28.969882 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d939684f7a845f7b267cf420779b603" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.971560 18688 raft_consensus.cc:695] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:28.987150 18688 consensus_queue.cc:237] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.987768 18509 leader_election.cc:304] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:28.988297 18511 leader_election.cc:304] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:28.988685 18664 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:28.974223 18395 raft_consensus.cc:2463] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:28.989248 18663 tablet_bootstrap.cc:492] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:28.989190 18746 raft_consensus.cc:2798] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.974661 18545 raft_consensus.cc:2463] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:28.974956 18689 consensus_queue.cc:1035] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:58:28.981801 18342 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:58:28.981696 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaf48f6743cb46e686eaa798a4a22957" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:28.990881 18393 raft_consensus.cc:3054] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:28.990978 18584 leader_election.cc:304] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:28.991930 18736 raft_consensus.cc:2798] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:28.992342 18736 raft_consensus.cc:695] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:28.993011 18736 consensus_queue.cc:237] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:28.995369 18689 raft_consensus.cc:491] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:28.995833 18689 raft_consensus.cc:513] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:28.982689 18732 leader_election.cc:290] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.983440 18292 catalog_manager.cc:5526] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:28.997777 18689 leader_election.cc:290] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:28.972172 18396 raft_consensus.cc:2463] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:28.878887 18661 consensus_queue.cc:260] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:28.999310 18393 raft_consensus.cc:2463] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:28.999329 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:28.999722 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:29.000089 18617 raft_consensus.cc:2463] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:29.000309 18396 raft_consensus.cc:2463] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:28.989879 18663 ts_tablet_manager.cc:1397] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.123s user 0.047s sys 0.055s
I20250112 20:58:28.989823 18746 raft_consensus.cc:695] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:29.000778 18719 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.001221 18435 leader_election.cc:304] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:29.001426 18746 consensus_queue.cc:237] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.002331 18689 raft_consensus.cc:2798] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.002770 18661 ts_tablet_manager.cc:1428] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.130s user 0.004s sys 0.002s
I20250112 20:58:29.002744 18689 raft_consensus.cc:491] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.007434 18663 raft_consensus.cc:357] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.008208 18663 raft_consensus.cc:383] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.008512 18663 raft_consensus.cc:738] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:28.989183 18664 ts_tablet_manager.cc:1397] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.141s user 0.015s sys 0.001s
I20250112 20:58:29.004135 18689 raft_consensus.cc:3054] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.010592 18294 catalog_manager.cc:5526] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 1 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:29.011142 18663 consensus_queue.cc:260] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.013448 18663 ts_tablet_manager.cc:1428] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250112 20:58:29.014321 18700 raft_consensus.cc:3054] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.003751 18661 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:29.001266 18719 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.021270 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.021832 18719 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
W20250112 20:58:29.022907 18362 leader_election.cc:343] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.023772 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:29.032799 18718 consensus_queue.cc:1035] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:29.015331 18664 raft_consensus.cc:357] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.014335 18663 tablet_bootstrap.cc:492] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:29.016187 18662 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:29.037045 18662 ts_tablet_manager.cc:1397] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.173s user 0.013s sys 0.017s
I20250112 20:58:29.039366 18662 raft_consensus.cc:357] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.040027 18662 raft_consensus.cc:383] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.040277 18662 raft_consensus.cc:738] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.029151 18290 catalog_manager.cc:5526] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:29.040871 18662 consensus_queue.cc:260] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.042076 18663 tablet_bootstrap.cc:654] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.042812 18662 ts_tablet_manager.cc:1428] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250112 20:58:29.043592 18662 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:29.044752 18664 raft_consensus.cc:383] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.045063 18664 raft_consensus.cc:738] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.045725 18664 consensus_queue.cc:260] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.031739 18689 raft_consensus.cc:513] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.047410 18661 tablet_bootstrap.cc:654] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.048003 18689 leader_election.cc:290] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:29.049885 18664 ts_tablet_manager.cc:1428] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.040s user 0.004s sys 0.001s
I20250112 20:58:29.050571 18664 tablet_bootstrap.cc:492] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:29.049533 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:29.054113 18395 raft_consensus.cc:3054] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.059554 18663 tablet_bootstrap.cc:492] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:29.059989 18663 ts_tablet_manager.cc:1397] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.046s user 0.011s sys 0.001s
I20250112 20:58:29.051770 18545 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:29.056756 18689 raft_consensus.cc:491] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.062577 18689 raft_consensus.cc:513] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.062971 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:29.063653 18617 raft_consensus.cc:3054] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.063786 18361 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38
I20250112 20:58:29.065070 18719 raft_consensus.cc:2798] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.065435 18719 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.065742 18719 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.071054 18662 tablet_bootstrap.cc:654] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.076695 18663 raft_consensus.cc:357] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.081146 18670 consensus_queue.cc:1035] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.082001 18664 tablet_bootstrap.cc:654] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.083777 18395 raft_consensus.cc:2463] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:29.085368 18435 leader_election.cc:304] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:29.086144 18718 raft_consensus.cc:2798] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:29.086565 18718 raft_consensus.cc:695] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:29.087293 18718 consensus_queue.cc:237] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.087654 18617 raft_consensus.cc:2463] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:29.077873 18663 raft_consensus.cc:383] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.091171 18663 raft_consensus.cc:738] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.101179 18663 consensus_queue.cc:260] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.103605 18663 ts_tablet_manager.cc:1428] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.043s user 0.008s sys 0.000s
I20250112 20:58:29.104185 18689 leader_election.cc:290] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:29.105381 18663 tablet_bootstrap.cc:492] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:29.104840 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6e68976f1b46278833369d2b5c0e95" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:29.112201 18700 raft_consensus.cc:2743] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.113821 18719 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:29.115760 18436 leader_election.cc:343] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.118425 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6e68976f1b46278833369d2b5c0e95" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:29.118629 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:29.124554 18670 raft_consensus.cc:491] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.125025 18670 raft_consensus.cc:513] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.125258 18719 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.126260 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:29.126829 18545 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.128748 18664 tablet_bootstrap.cc:492] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:29.122309 18471 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.129205 18664 ts_tablet_manager.cc:1397] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.079s user 0.015s sys 0.013s
I20250112 20:58:29.129570 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:29.130231 18435 leader_election.cc:343] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.130689 18435 leader_election.cc:304] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:29.131769 18718 raft_consensus.cc:2743] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.132618 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:29.133361 18617 raft_consensus.cc:2463] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:29.131670 18664 raft_consensus.cc:357] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:29.131801 18362 leader_election.cc:343] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.134392 18664 raft_consensus.cc:383] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.134711 18664 raft_consensus.cc:738] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.134497 18360 leader_election.cc:304] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38
I20250112 20:58:29.135354 18664 consensus_queue.cc:260] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.126901 18670 leader_election.cc:290] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:29.136315 18670 raft_consensus.cc:2798] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.136683 18670 raft_consensus.cc:491] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.137010 18670 raft_consensus.cc:3054] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.136520 18294 catalog_manager.cc:5526] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:29.137632 18664 ts_tablet_manager.cc:1428] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20250112 20:58:29.138553 18664 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:29.145757 18661 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:29.146101 18661 ts_tablet_manager.cc:1397] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.143s user 0.020s sys 0.009s
I20250112 20:58:29.149963 18661 raft_consensus.cc:357] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.150583 18661 raft_consensus.cc:383] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.150872 18663 tablet_bootstrap.cc:654] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.150997 18661 raft_consensus.cc:738] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.152052 18661 consensus_queue.cc:260] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.152904 18736 raft_consensus.cc:491] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.154001 18736 raft_consensus.cc:513] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.155982 18736 leader_election.cc:290] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.157487 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1906b96af18e4a7184907ae1b525aa02" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:29.160220 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1906b96af18e4a7184907ae1b525aa02" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.161469 18585 leader_election.cc:304] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:29.162192 18736 raft_consensus.cc:2743] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.163290 18664 tablet_bootstrap.cc:654] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.163786 18545 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:29.164875 18361 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:29.167281 18662 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:29.167699 18662 ts_tablet_manager.cc:1397] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.124s user 0.007s sys 0.004s
I20250112 20:58:29.168252 18718 consensus_queue.cc:1035] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.165201 18719 consensus_queue.cc:1035] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20250112 20:58:29.173542 18662 raft_consensus.cc:357] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.174178 18662 raft_consensus.cc:383] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.174427 18719 raft_consensus.cc:2798] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:29.174860 18719 raft_consensus.cc:695] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:29.175655 18719 consensus_queue.cc:237] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.177965 18619 raft_consensus.cc:3054] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.182299 18661 ts_tablet_manager.cc:1428] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.036s user 0.005s sys 0.001s
I20250112 20:58:29.183236 18661 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:29.184792 18670 raft_consensus.cc:513] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.187613 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:29.187889 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:29.188218 18618 raft_consensus.cc:3054] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.174419 18662 raft_consensus.cc:738] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.183547 18293 catalog_manager.cc:5526] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:29.191350 18662 consensus_queue.cc:260] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.192215 18470 raft_consensus.cc:3054] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.192319 18664 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:29.192992 18664 ts_tablet_manager.cc:1397] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.055s user 0.018s sys 0.006s
I20250112 20:58:29.193140 18768 raft_consensus.cc:491] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.193574 18768 raft_consensus.cc:513] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.188182 18471 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:29.196583 18768 leader_election.cc:290] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.201050 18470 raft_consensus.cc:2463] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:29.202906 18362 leader_election.cc:304] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:29.204085 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f0a2bc717d404f8f3afd03c3d5a60c" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.204576 18766 raft_consensus.cc:2798] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
W20250112 20:58:29.205478 18362 leader_election.cc:343] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.206590 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f0a2bc717d404f8f3afd03c3d5a60c" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:29.206624 18700 consensus_queue.cc:1035] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:29.190686 18670 leader_election.cc:290] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
W20250112 20:58:29.209901 18361 leader_election.cc:343] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.210325 18361 leader_election.cc:304] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:29.202782 18618 raft_consensus.cc:2463] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:29.213475 18619 raft_consensus.cc:3054] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.223992 18670 consensus_queue.cc:1035] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:29.228528 18662 ts_tablet_manager.cc:1428] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.061s user 0.006s sys 0.001s
I20250112 20:58:29.242067 18662 tablet_bootstrap.cc:492] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:29.243456 18766 raft_consensus.cc:695] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:29.220333 18776 raft_consensus.cc:2743] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.244468 18736 raft_consensus.cc:491] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.245036 18736 raft_consensus.cc:513] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.244297 18766 consensus_queue.cc:237] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.247066 18736 leader_election.cc:290] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:29.246914 18689 consensus_queue.cc:1035] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.249483 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:29.250249 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:29.250785 18583 leader_election.cc:343] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:29.251588 18585 leader_election.cc:343] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.252007 18585 leader_election.cc:304] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:29.253108 18736 raft_consensus.cc:2743] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.240343 18661 tablet_bootstrap.cc:654] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.255856 18292 catalog_manager.cc:5526] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:29.270542 18663 tablet_bootstrap.cc:492] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:29.271224 18663 ts_tablet_manager.cc:1397] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.166s user 0.010s sys 0.016s
I20250112 20:58:29.271749 18732 consensus_queue.cc:1035] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.273751 18663 raft_consensus.cc:357] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.274451 18663 raft_consensus.cc:383] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.274696 18663 raft_consensus.cc:738] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.275425 18663 consensus_queue.cc:260] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.278307 18663 ts_tablet_manager.cc:1428] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:58:29.279497 18663 tablet_bootstrap.cc:492] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:29.235199 18719 consensus_queue.cc:1035] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
I20250112 20:58:29.291986 18663 tablet_bootstrap.cc:654] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.237663 18664 raft_consensus.cc:357] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.295357 18664 raft_consensus.cc:383] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.295708 18664 raft_consensus.cc:738] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.296646 18664 consensus_queue.cc:260] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.300143 18700 consensus_queue.cc:1035] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:29.307781 18663 tablet_bootstrap.cc:492] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:29.309898 18662 tablet_bootstrap.cc:654] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.312214 18663 ts_tablet_manager.cc:1397] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.009s
I20250112 20:58:29.315229 18663 raft_consensus.cc:357] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.316365 18661 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:29.317013 18661 ts_tablet_manager.cc:1397] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.134s user 0.009s sys 0.003s
I20250112 20:58:29.319413 18661 raft_consensus.cc:357] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.320099 18661 raft_consensus.cc:383] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.320355 18661 raft_consensus.cc:738] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.319429 18664 ts_tablet_manager.cc:1428] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.126s user 0.005s sys 0.000s
I20250112 20:58:29.320968 18661 consensus_queue.cc:260] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.321615 18664 tablet_bootstrap.cc:492] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:29.326309 18668 consensus_queue.cc:1035] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.328552 18689 consensus_queue.cc:1035] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:29.338205 18664 tablet_bootstrap.cc:654] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.316403 18663 raft_consensus.cc:383] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.323133 18661 ts_tablet_manager.cc:1428] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20250112 20:58:29.339705 18663 raft_consensus.cc:738] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.340533 18661 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:29.340452 18663 consensus_queue.cc:260] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.342573 18663 ts_tablet_manager.cc:1428] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20250112 20:58:29.343355 18663 tablet_bootstrap.cc:492] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:29.347075 18661 tablet_bootstrap.cc:654] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.348225 18668 raft_consensus.cc:491] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.348709 18668 raft_consensus.cc:513] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.351941 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec5696903d64121bab105da9eb5a3b9" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.353021 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec5696903d64121bab105da9eb5a3b9" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:29.353704 18585 leader_election.cc:343] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:29.354420 18584 leader_election.cc:343] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.354838 18584 leader_election.cc:304] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:29.355656 18732 raft_consensus.cc:2743] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.356227 18668 leader_election.cc:290] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.357358 18774 raft_consensus.cc:491] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.358001 18774 raft_consensus.cc:513] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.360417 18774 leader_election.cc:290] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:29.362114 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:29.363457 18361 leader_election.cc:343] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.364096 18663 tablet_bootstrap.cc:654] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.365051 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:29.366817 18360 leader_election.cc:343] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.367389 18360 leader_election.cc:304] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:29.368865 18774 raft_consensus.cc:2743] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.382685 18688 raft_consensus.cc:491] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.383229 18688 raft_consensus.cc:513] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.385402 18688 leader_election.cc:290] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:29.387863 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c257b2d0772f4078b2ca2d7c1298e4ae" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:29.390164 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c257b2d0772f4078b2ca2d7c1298e4ae" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.391772 18511 leader_election.cc:304] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:29.392927 18688 raft_consensus.cc:2743] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.393729 18292 catalog_manager.cc:5526] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:29.397343 18545 raft_consensus.cc:3054] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.406572 18688 consensus_queue.cc:1035] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.395413 18663 tablet_bootstrap.cc:492] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:29.411697 18689 consensus_queue.cc:1035] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.413408 18663 ts_tablet_manager.cc:1397] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.000s
I20250112 20:58:29.414852 18668 consensus_queue.cc:1035] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:29.416056 18662 tablet_bootstrap.cc:492] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:29.416496 18662 ts_tablet_manager.cc:1397] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.175s user 0.032s sys 0.051s
I20250112 20:58:29.417140 18663 raft_consensus.cc:357] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.417819 18663 raft_consensus.cc:383] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.418074 18663 raft_consensus.cc:738] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.418699 18662 raft_consensus.cc:357] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.418664 18663 consensus_queue.cc:260] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.419493 18662 raft_consensus.cc:383] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.419822 18662 raft_consensus.cc:738] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.420388 18662 consensus_queue.cc:260] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.427642 18662 ts_tablet_manager.cc:1428] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250112 20:58:29.428540 18662 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:29.433974 18662 tablet_bootstrap.cc:654] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.434830 18664 tablet_bootstrap.cc:492] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:29.435420 18664 ts_tablet_manager.cc:1397] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.114s user 0.010s sys 0.000s
I20250112 20:58:29.440441 18663 ts_tablet_manager.cc:1428] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20250112 20:58:29.441367 18663 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:29.445071 18746 consensus_queue.cc:1035] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.446615 18663 tablet_bootstrap.cc:654] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.450291 18662 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:29.450886 18662 ts_tablet_manager.cc:1397] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.003s
I20250112 20:58:29.455358 18662 raft_consensus.cc:357] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.456234 18662 raft_consensus.cc:383] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.456557 18662 raft_consensus.cc:738] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.457347 18662 consensus_queue.cc:260] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.460604 18662 ts_tablet_manager.cc:1428] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250112 20:58:29.461778 18662 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:29.469055 18662 tablet_bootstrap.cc:654] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.477886 18668 consensus_queue.cc:1035] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.488006 18661 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:29.488173 18662 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:29.488489 18661 ts_tablet_manager.cc:1397] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.148s user 0.029s sys 0.043s
I20250112 20:58:29.488583 18662 ts_tablet_manager.cc:1397] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.005s
I20250112 20:58:29.491012 18661 raft_consensus.cc:357] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.491816 18661 raft_consensus.cc:383] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.492331 18661 raft_consensus.cc:738] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.493063 18661 consensus_queue.cc:260] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.495187 18661 ts_tablet_manager.cc:1428] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:58:29.496675 18661 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:29.495031 18664 raft_consensus.cc:357] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.498876 18664 raft_consensus.cc:383] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.499174 18664 raft_consensus.cc:738] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.491688 18662 raft_consensus.cc:357] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.499816 18662 raft_consensus.cc:383] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.500115 18662 raft_consensus.cc:738] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.500769 18662 consensus_queue.cc:260] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.502286 18663 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:29.502596 18661 tablet_bootstrap.cc:654] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.502753 18663 ts_tablet_manager.cc:1397] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.062s user 0.019s sys 0.006s
I20250112 20:58:29.503007 18662 ts_tablet_manager.cc:1428] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20250112 20:58:29.503959 18662 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:29.505735 18689 raft_consensus.cc:491] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.506217 18689 raft_consensus.cc:513] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.505990 18664 consensus_queue.cc:260] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.508167 18689 leader_election.cc:290] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.509167 18664 ts_tablet_manager.cc:1428] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.073s user 0.009s sys 0.000s
I20250112 20:58:29.512863 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee08d81c32b648e085f17dd937f5c512" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:29.514163 18436 leader_election.cc:343] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.515152 18662 tablet_bootstrap.cc:654] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.517032 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee08d81c32b648e085f17dd937f5c512" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:29.517683 18396 raft_consensus.cc:2463] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:29.519011 18435 leader_election.cc:304] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:29.515918 18664 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:29.523751 18689 raft_consensus.cc:2798] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.524296 18689 raft_consensus.cc:491] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.525035 18689 raft_consensus.cc:3054] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.505241 18663 raft_consensus.cc:357] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.528198 18663 raft_consensus.cc:383] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.529098 18663 raft_consensus.cc:738] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.537348 18663 consensus_queue.cc:260] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.540495 18664 tablet_bootstrap.cc:654] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.544440 18662 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:29.544991 18662 ts_tablet_manager.cc:1397] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.041s user 0.018s sys 0.001s
I20250112 20:58:29.547447 18662 raft_consensus.cc:357] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.548266 18662 raft_consensus.cc:383] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.548569 18662 raft_consensus.cc:738] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.549263 18662 consensus_queue.cc:260] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.550278 18746 raft_consensus.cc:491] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.549799 18689 raft_consensus.cc:513] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.550743 18746 raft_consensus.cc:513] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.552771 18746 leader_election.cc:290] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:29.555102 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee08d81c32b648e085f17dd937f5c512" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:29.555685 18396 raft_consensus.cc:3054] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.556026 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec5696903d64121bab105da9eb5a3b9" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:29.556666 18616 raft_consensus.cc:2463] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:29.558311 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee08d81c32b648e085f17dd937f5c512" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:29.559230 18663 ts_tablet_manager.cc:1428] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.056s user 0.009s sys 0.001s
I20250112 20:58:29.559937 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec5696903d64121bab105da9eb5a3b9" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.560428 18663 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:29.560585 18471 raft_consensus.cc:2463] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:29.561941 18509 leader_election.cc:304] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
W20250112 20:58:29.564033 18436 leader_election.cc:343] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.564435 18396 raft_consensus.cc:2463] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:29.564769 18746 raft_consensus.cc:2798] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.565227 18746 raft_consensus.cc:491] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.565629 18746 raft_consensus.cc:3054] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.552385 18689 leader_election.cc:290] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.567494 18661 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:29.567950 18661 ts_tablet_manager.cc:1397] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.071s user 0.025s sys 0.040s
I20250112 20:58:29.569573 18663 tablet_bootstrap.cc:654] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.570878 18664 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:29.571334 18664 ts_tablet_manager.cc:1397] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.056s user 0.016s sys 0.003s
I20250112 20:58:29.573079 18662 ts_tablet_manager.cc:1428] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.028s user 0.002s sys 0.004s
I20250112 20:58:29.573817 18774 consensus_queue.cc:1035] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.574385 18664 raft_consensus.cc:357] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.576445 18664 raft_consensus.cc:383] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.576906 18664 raft_consensus.cc:738] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.576282 18661 raft_consensus.cc:357] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.577476 18661 raft_consensus.cc:383] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.577893 18661 raft_consensus.cc:738] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.577634 18664 consensus_queue.cc:260] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.578717 18661 consensus_queue.cc:260] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.574702 18746 raft_consensus.cc:513] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.581204 18662 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:29.582264 18746 leader_election.cc:290] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:29.582474 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec5696903d64121bab105da9eb5a3b9" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:29.583083 18471 raft_consensus.cc:3054] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.586526 18661 ts_tablet_manager.cc:1428] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.018s user 0.009s sys 0.000s
I20250112 20:58:29.586637 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec5696903d64121bab105da9eb5a3b9" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:29.587334 18616 raft_consensus.cc:3054] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.587848 18661 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:29.594672 18746 consensus_queue.cc:1035] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.595024 18435 leader_election.cc:304] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:29.595662 18661 tablet_bootstrap.cc:654] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.596028 18689 raft_consensus.cc:2798] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:29.596261 18668 raft_consensus.cc:491] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.596581 18689 raft_consensus.cc:695] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:29.596769 18668 raft_consensus.cc:513] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.597465 18689 consensus_queue.cc:237] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.598726 18668 leader_election.cc:290] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.601455 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6e68976f1b46278833369d2b5c0e95" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.601984 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6e68976f1b46278833369d2b5c0e95" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:29.602448 18469 raft_consensus.cc:2463] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:29.607827 18585 leader_election.cc:304] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
W20250112 20:58:29.608716 18584 leader_election.cc:343] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.580356 18664 ts_tablet_manager.cc:1428] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20250112 20:58:29.612031 18664 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:29.614403 18668 raft_consensus.cc:2798] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.614806 18668 raft_consensus.cc:491] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.615173 18668 raft_consensus.cc:3054] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.614910 18746 consensus_queue.cc:1035] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.632660 18616 raft_consensus.cc:2463] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:29.634392 18509 leader_election.cc:304] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:29.635426 18746 raft_consensus.cc:2798] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:29.636013 18746 raft_consensus.cc:695] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:29.637049 18746 consensus_queue.cc:237] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.646427 18664 tablet_bootstrap.cc:654] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.650527 18471 raft_consensus.cc:2463] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:29.652020 18757 consensus_queue.cc:1035] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:29.686139 18732 raft_consensus.cc:491] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.687892 18732 raft_consensus.cc:513] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.689874 18732 leader_election.cc:290] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.696256 18661 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:29.696740 18661 ts_tablet_manager.cc:1397] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.109s user 0.020s sys 0.005s
I20250112 20:58:29.701813 18732 raft_consensus.cc:491] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.702294 18732 raft_consensus.cc:513] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.704180 18732 leader_election.cc:290] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:29.705471 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:29.705775 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.706949 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "684a3c0a345f4ce483768dfe1572ca13" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
W20250112 20:58:29.709200 18584 leader_election.cc:343] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.699229 18757 raft_consensus.cc:491] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.710091 18757 raft_consensus.cc:513] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.713018 18662 tablet_bootstrap.cc:654] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.715255 18663 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:29.715773 18663 ts_tablet_manager.cc:1397] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.156s user 0.054s sys 0.059s
I20250112 20:58:29.715178 18757 leader_election.cc:290] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
W20250112 20:58:29.717820 18583 leader_election.cc:343] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:29.718475 18585 leader_election.cc:343] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.718900 18585 leader_election.cc:304] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:29.719630 18736 raft_consensus.cc:2743] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:29.718327 18663 raft_consensus.cc:357] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.721083 18663 raft_consensus.cc:383] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.721355 18663 raft_consensus.cc:738] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.722193 18663 consensus_queue.cc:260] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.725404 18663 ts_tablet_manager.cc:1428] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20250112 20:58:29.726241 18663 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:29.726665 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f0a2bc717d404f8f3afd03c3d5a60c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:29.727509 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f0a2bc717d404f8f3afd03c3d5a60c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
W20250112 20:58:29.727874 18436 leader_election.cc:343] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.728137 18396 raft_consensus.cc:2463] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:29.729411 18435 leader_election.cc:304] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:29.685792 18668 raft_consensus.cc:513] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.731478 18663 tablet_bootstrap.cc:654] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.731958 18668 leader_election.cc:290] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.732335 18776 consensus_queue.cc:1035] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.732991 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6e68976f1b46278833369d2b5c0e95" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
W20250112 20:58:29.734498 18584 leader_election.cc:343] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.735342 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6e68976f1b46278833369d2b5c0e95" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:29.735179 18661 raft_consensus.cc:357] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.735965 18469 raft_consensus.cc:3054] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.736181 18661 raft_consensus.cc:383] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.737553 18661 raft_consensus.cc:738] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.738632 18661 consensus_queue.cc:260] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.741111 18718 raft_consensus.cc:2798] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.745743 18718 raft_consensus.cc:491] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.746145 18718 raft_consensus.cc:3054] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.748858 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "684a3c0a345f4ce483768dfe1572ca13" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:29.749716 18544 raft_consensus.cc:2463] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:29.750828 18469 raft_consensus.cc:2463] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:29.751348 18584 leader_election.cc:304] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:29.752244 18585 leader_election.cc:304] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:29.753737 18732 raft_consensus.cc:2798] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.754181 18732 raft_consensus.cc:491] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.754570 18732 raft_consensus.cc:3054] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.756798 18291 catalog_manager.cc:5526] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:29.761025 18668 raft_consensus.cc:2798] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:29.761955 18668 raft_consensus.cc:695] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:29.767395 18661 ts_tablet_manager.cc:1428] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.070s user 0.004s sys 0.003s
I20250112 20:58:29.774056 18661 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:29.775305 18670 consensus_queue.cc:1035] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20250112 20:58:29.763089 18668 consensus_queue.cc:237] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:29.792683 18294 catalog_manager.cc:5526] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:29.800419 18661 tablet_bootstrap.cc:654] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.819880 18664 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:29.820474 18664 ts_tablet_manager.cc:1397] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.209s user 0.026s sys 0.019s
I20250112 20:58:29.827389 18663 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:29.827930 18663 ts_tablet_manager.cc:1397] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.102s user 0.018s sys 0.017s
I20250112 20:58:29.830361 18663 raft_consensus.cc:357] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.831030 18663 raft_consensus.cc:383] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.822219 18689 consensus_queue.cc:1035] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20250112 20:58:29.831600 18663 raft_consensus.cc:738] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.832223 18663 consensus_queue.cc:260] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.833894 18664 raft_consensus.cc:357] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.834789 18664 raft_consensus.cc:383] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.835088 18664 raft_consensus.cc:738] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.835647 18664 consensus_queue.cc:260] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.837179 18664 ts_tablet_manager.cc:1428] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20250112 20:58:29.837962 18664 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:29.837718 18732 raft_consensus.cc:513] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.834753 18663 ts_tablet_manager.cc:1428] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20250112 20:58:29.839569 18663 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:29.840876 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "684a3c0a345f4ce483768dfe1572ca13" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:29.841590 18544 raft_consensus.cc:3054] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.855708 18663 tablet_bootstrap.cc:654] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.866101 18732 leader_election.cc:290] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:29.870003 18664 tablet_bootstrap.cc:654] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:29.836050 18718 raft_consensus.cc:513] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.868398 18670 raft_consensus.cc:491] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.888257 18757 raft_consensus.cc:491] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.888818 18757 raft_consensus.cc:513] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.898757 18662 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:29.899382 18662 ts_tablet_manager.cc:1397] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.318s user 0.013s sys 0.007s
I20250112 20:58:29.888919 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "684a3c0a345f4ce483768dfe1572ca13" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:29.887385 18670 raft_consensus.cc:513] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:29.927201 18583 leader_election.cc:343] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:29.922327 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 126872us
I20250112 20:58:29.939428 18757 leader_election.cc:290] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.939545 18662 raft_consensus.cc:357] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.941681 18670 leader_election.cc:290] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:29.942472 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:29.940528 18662 raft_consensus.cc:383] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.945140 18662 raft_consensus.cc:738] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.945923 18662 consensus_queue.cc:260] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:29.948622 18661 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:29.949277 18661 ts_tablet_manager.cc:1397] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.176s user 0.018s sys 0.008s
W20250112 20:58:29.950474 18360 leader_election.cc:343] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:29.952373 18544 raft_consensus.cc:2463] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:29.955924 18718 leader_election.cc:290] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:29.957707 18584 leader_election.cc:304] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:29.963979 18662 ts_tablet_manager.cc:1428] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.064s user 0.007s sys 0.002s
I20250112 20:58:29.964691 18662 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:29.966427 18774 consensus_queue.cc:1035] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:29.968367 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:29.969028 18469 raft_consensus.cc:2463] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:29.971788 18362 leader_election.cc:304] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:29.972035 18661 raft_consensus.cc:357] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.972916 18719 raft_consensus.cc:2798] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:29.973276 18719 raft_consensus.cc:491] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:29.973606 18719 raft_consensus.cc:3054] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.976421 18293 catalog_manager.cc:5526] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 1 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:29.979326 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f0a2bc717d404f8f3afd03c3d5a60c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:29.979712 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f8bbae0e0c494ab580a5f2338ef4ea" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:29.979992 18396 raft_consensus.cc:3054] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:29.959932 18668 raft_consensus.cc:2798] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:29.972939 18661 raft_consensus.cc:383] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:29.982975 18661 raft_consensus.cc:738] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:29.984133 18661 consensus_queue.cc:260] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:29.985237 18746 raft_consensus.cc:491] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:29.986131 18668 raft_consensus.cc:695] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:29.985898 18393 raft_consensus.cc:2463] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:29.999186 18663 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:29.986701 18746 raft_consensus.cc:513] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:29.999622 18663 ts_tablet_manager.cc:1397] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.160s user 0.017s sys 0.015s
I20250112 20:58:30.000819 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f8bbae0e0c494ab580a5f2338ef4ea" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.001777 18746 leader_election.cc:290] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:30.003662 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.003468 18663 raft_consensus.cc:357] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.004302 18663 raft_consensus.cc:383] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.004902 18663 raft_consensus.cc:738] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.005793 18663 consensus_queue.cc:260] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.007665 18663 ts_tablet_manager.cc:1428] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20250112 20:58:30.008560 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:30.009181 18619 raft_consensus.cc:2463] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:30.010342 18509 leader_election.cc:304] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:30.004328 18394 raft_consensus.cc:2463] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:30.014201 18746 raft_consensus.cc:2798] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:30.014559 18746 raft_consensus.cc:491] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:30.014899 18746 raft_consensus.cc:3054] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.017122 18664 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:30.017607 18664 ts_tablet_manager.cc:1397] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.180s user 0.016s sys 0.031s
I20250112 20:58:30.017867 18668 consensus_queue.cc:237] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.018580 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f0a2bc717d404f8f3afd03c3d5a60c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:30.020051 18664 raft_consensus.cc:357] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.020776 18664 raft_consensus.cc:383] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.021035 18664 raft_consensus.cc:738] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.021229 18746 raft_consensus.cc:513] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.021806 18664 consensus_queue.cc:260] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.023375 18435 leader_election.cc:304] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:30.024318 18718 raft_consensus.cc:2798] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:30.024716 18718 raft_consensus.cc:491] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:30.025085 18718 raft_consensus.cc:3054] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.027446 18661 ts_tablet_manager.cc:1428] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.078s user 0.007s sys 0.000s
I20250112 20:58:30.030108 18661 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:30.031664 18774 raft_consensus.cc:491] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.031852 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:30.032061 18774 raft_consensus.cc:513] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.032514 18619 raft_consensus.cc:3054] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.033753 18664 ts_tablet_manager.cc:1428] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20250112 20:58:30.034312 18774 leader_election.cc:290] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:30.034060 18719 raft_consensus.cc:513] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.035329 18664 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:30.034199 18746 leader_election.cc:290] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:30.036770 18396 raft_consensus.cc:2463] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:30.038713 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:30.039027 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:30.039482 18617 raft_consensus.cc:2463] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:30.040263 18718 raft_consensus.cc:513] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:30.040791 18360 leader_election.cc:343] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.041179 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:30.041775 18393 raft_consensus.cc:3054] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.041553 18360 leader_election.cc:304] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:30.043900 18435 leader_election.cc:304] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:30.045071 18821 raft_consensus.cc:2798] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:30.045606 18664 tablet_bootstrap.cc:654] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.045534 18821 raft_consensus.cc:695] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:30.046592 18821 consensus_queue.cc:237] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.048033 18661 tablet_bootstrap.cc:654] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.048156 18393 raft_consensus.cc:2463] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:30.049008 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:30.049636 18470 raft_consensus.cc:3054] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.050236 18774 raft_consensus.cc:2798] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:30.050740 18774 raft_consensus.cc:491] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:30.051152 18774 raft_consensus.cc:3054] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.055842 18470 raft_consensus.cc:2463] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:30.056859 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.057592 18544 raft_consensus.cc:2463] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:30.064565 18619 raft_consensus.cc:2463] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:30.066025 18509 leader_election.cc:304] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:30.066783 18746 raft_consensus.cc:2798] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:30.067265 18746 raft_consensus.cc:695] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:30.068017 18746 consensus_queue.cc:237] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.069216 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f8bbae0e0c494ab580a5f2338ef4ea" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:30.070024 18396 raft_consensus.cc:3054] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.071843 18663 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:30.075399 18662 tablet_bootstrap.cc:654] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.075732 18362 leader_election.cc:304] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:30.076484 18776 raft_consensus.cc:2798] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:30.076956 18776 raft_consensus.cc:695] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:30.077764 18776 consensus_queue.cc:237] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.078537 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f8bbae0e0c494ab580a5f2338ef4ea" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:30.042109 18718 leader_election.cc:290] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:30.089177 18719 leader_election.cc:290] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:30.089820 18718 consensus_queue.cc:1035] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:30.099655 18663 tablet_bootstrap.cc:654] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
W20250112 20:58:30.101709 18436 leader_election.cc:343] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:30.103168 18436 leader_election.cc:343] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.103627 18294 catalog_manager.cc:5526] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
W20250112 20:58:30.104329 18436 leader_election.cc:343] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.117755 18774 raft_consensus.cc:513] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.119705 18774 leader_election.cc:290] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:30.120435 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:30.121114 18544 raft_consensus.cc:3054] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.121840 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:30.122598 18619 raft_consensus.cc:3054] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.125202 18293 catalog_manager.cc:5526] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 1 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:30.126879 18396 raft_consensus.cc:2463] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:30.126896 18292 catalog_manager.cc:5526] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:30.138964 18668 raft_consensus.cc:491] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.139654 18668 raft_consensus.cc:513] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.143402 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.131629 18746 raft_consensus.cc:491] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.146040 18746 raft_consensus.cc:513] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.149401 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:30.150385 18746 leader_election.cc:290] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
W20250112 20:58:30.152172 18584 leader_election.cc:343] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.152472 18435 leader_election.cc:304] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
W20250112 20:58:30.153548 18509 leader_election.cc:343] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.153576 18718 raft_consensus.cc:2798] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:30.157593 18718 raft_consensus.cc:695] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:30.158619 18718 consensus_queue.cc:237] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.161942 18544 raft_consensus.cc:2463] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:30.162382 18812 consensus_queue.cc:1035] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:30.163050 18361 leader_election.cc:304] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:30.163973 18774 raft_consensus.cc:2798] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:30.165048 18774 raft_consensus.cc:695] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:30.173702 18774 consensus_queue.cc:237] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.177378 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.181552 18661 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:30.181936 18661 ts_tablet_manager.cc:1397] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.152s user 0.031s sys 0.023s
I20250112 20:58:30.171262 18619 raft_consensus.cc:2463] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:30.183445 18668 leader_election.cc:290] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
W20250112 20:58:30.190568 18509 leader_election.cc:343] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.191051 18509 leader_election.cc:304] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:30.192889 18661 raft_consensus.cc:357] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.193687 18661 raft_consensus.cc:383] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.193987 18661 raft_consensus.cc:738] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.194986 18746 raft_consensus.cc:2743] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.195843 18661 consensus_queue.cc:260] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.198200 18661 ts_tablet_manager.cc:1428] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.016s user 0.004s sys 0.002s
I20250112 20:58:30.199136 18661 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:30.200568 18436 consensus_peers.cc:487] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:30.201454 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
W20250112 20:58:30.203678 18583 leader_election.cc:343] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.204320 18583 leader_election.cc:304] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:30.216354 18668 raft_consensus.cc:2743] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.226828 18662 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:30.227366 18662 ts_tablet_manager.cc:1397] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.263s user 0.016s sys 0.016s
I20250112 20:58:30.232842 18663 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:30.233310 18663 ts_tablet_manager.cc:1397] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.162s user 0.022s sys 0.033s
I20250112 20:58:30.242197 18812 consensus_queue.cc:1035] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:30.242245 18662 raft_consensus.cc:357] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.243150 18662 raft_consensus.cc:383] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.243489 18662 raft_consensus.cc:738] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.244102 18662 consensus_queue.cc:260] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.247498 18662 ts_tablet_manager.cc:1428] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.020s user 0.001s sys 0.005s
I20250112 20:58:30.248389 18662 tablet_bootstrap.cc:492] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:30.249598 18292 catalog_manager.cc:5526] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:30.251055 18664 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:30.251513 18664 ts_tablet_manager.cc:1397] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.216s user 0.028s sys 0.069s
I20250112 20:58:30.236502 18663 raft_consensus.cc:357] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.252871 18663 raft_consensus.cc:383] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.253080 18663 raft_consensus.cc:738] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.253592 18663 consensus_queue.cc:260] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:30.254673 18342 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:58:30.255788 18664 raft_consensus.cc:357] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.256467 18664 raft_consensus.cc:383] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.256740 18664 raft_consensus.cc:738] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.257055 18662 tablet_bootstrap.cc:654] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.257378 18664 consensus_queue.cc:260] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.259491 18664 ts_tablet_manager.cc:1428] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20250112 20:58:30.260511 18664 tablet_bootstrap.cc:492] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:30.267796 18661 tablet_bootstrap.cc:654] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.269250 18663 ts_tablet_manager.cc:1428] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.036s user 0.007s sys 0.000s
I20250112 20:58:30.272879 18663 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:30.281888 18664 tablet_bootstrap.cc:654] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
W20250112 20:58:30.297308 18584 consensus_peers.cc:487] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:30.297175 18290 catalog_manager.cc:5526] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:30.301961 18664 tablet_bootstrap.cc:492] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:30.302832 18663 tablet_bootstrap.cc:654] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.302915 18664 ts_tablet_manager.cc:1397] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.043s user 0.011s sys 0.008s
I20250112 20:58:30.303879 18662 tablet_bootstrap.cc:492] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:30.303860 18668 raft_consensus.cc:491] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.304453 18662 ts_tablet_manager.cc:1397] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.056s user 0.015s sys 0.016s
I20250112 20:58:30.304453 18668 raft_consensus.cc:513] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.305437 18664 raft_consensus.cc:357] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.306272 18664 raft_consensus.cc:383] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.306531 18664 raft_consensus.cc:738] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.307183 18664 consensus_queue.cc:260] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.308115 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.307983 18662 raft_consensus.cc:357] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.308852 18662 raft_consensus.cc:383] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.309190 18662 raft_consensus.cc:738] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.310111 18662 consensus_queue.cc:260] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:30.310873 18583 leader_election.cc:343] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.312278 18662 ts_tablet_manager.cc:1428] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:58:30.313380 18662 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:30.318346 18732 consensus_queue.cc:1035] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:30.319676 18662 tablet_bootstrap.cc:654] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.320472 18663 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:30.320966 18663 ts_tablet_manager.cc:1397] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.048s user 0.013s sys 0.000s
I20250112 20:58:30.321678 18664 ts_tablet_manager.cc:1428] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20250112 20:58:30.323149 18668 leader_election.cc:290] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:30.324553 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:30.326404 18585 leader_election.cc:343] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.326867 18585 leader_election.cc:304] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:30.327697 18732 raft_consensus.cc:2743] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.330308 18664 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:30.323666 18663 raft_consensus.cc:357] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.331211 18663 raft_consensus.cc:383] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.331507 18663 raft_consensus.cc:738] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.333648 18663 consensus_queue.cc:260] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.336129 18664 tablet_bootstrap.cc:654] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.338001 18662 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:30.338519 18662 ts_tablet_manager.cc:1397] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.012s
I20250112 20:58:30.338449 18732 raft_consensus.cc:491] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.339114 18732 raft_consensus.cc:513] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.341460 18662 raft_consensus.cc:357] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.341960 18732 leader_election.cc:290] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:30.342052 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737c8961a9424363b63a9688853bea1a" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.342319 18662 raft_consensus.cc:383] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.342934 18662 raft_consensus.cc:738] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.343607 18663 ts_tablet_manager.cc:1428] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
W20250112 20:58:30.343866 18583 leader_election.cc:343] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.343739 18662 consensus_queue.cc:260] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.344483 18663 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:30.346122 18662 ts_tablet_manager.cc:1428] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:58:30.346529 18746 raft_consensus.cc:491] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.347179 18662 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:30.347172 18746 raft_consensus.cc:513] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.350606 18746 leader_election.cc:290] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:30.351722 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6e68976f1b46278833369d2b5c0e95" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:30.354058 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6e68976f1b46278833369d2b5c0e95" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:30.354732 18662 tablet_bootstrap.cc:654] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.355368 18663 tablet_bootstrap.cc:654] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.355634 18511 leader_election.cc:304] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:30.357340 18746 raft_consensus.cc:2743] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.346510 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737c8961a9424363b63a9688853bea1a" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:30.359723 18585 leader_election.cc:343] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.360322 18585 leader_election.cc:304] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:30.361264 18732 raft_consensus.cc:2743] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.380178 18663 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:30.380980 18662 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:30.381306 18663 ts_tablet_manager.cc:1397] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.002s
I20250112 20:58:30.381656 18662 ts_tablet_manager.cc:1397] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.035s user 0.015s sys 0.006s
I20250112 20:58:30.384714 18662 raft_consensus.cc:357] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.385435 18662 raft_consensus.cc:383] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.385727 18662 raft_consensus.cc:738] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.386499 18662 consensus_queue.cc:260] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.389241 18662 ts_tablet_manager.cc:1428] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:58:30.390244 18662 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:30.390738 18663 raft_consensus.cc:357] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.391470 18663 raft_consensus.cc:383] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.391731 18663 raft_consensus.cc:738] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.394721 18663 consensus_queue.cc:260] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.397212 18662 tablet_bootstrap.cc:654] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.400204 18663 ts_tablet_manager.cc:1428] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.019s user 0.005s sys 0.002s
I20250112 20:58:30.405376 18661 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:30.372097 18290 catalog_manager.cc:5526] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:30.406015 18661 ts_tablet_manager.cc:1397] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.207s user 0.024s sys 0.025s
I20250112 20:58:30.408622 18661 raft_consensus.cc:357] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.409301 18661 raft_consensus.cc:383] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.410379 18663 tablet_bootstrap.cc:492] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:30.412184 18662 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:30.412662 18662 ts_tablet_manager.cc:1397] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.007s
I20250112 20:58:30.414621 18668 raft_consensus.cc:491] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.415047 18668 raft_consensus.cc:513] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.415444 18662 raft_consensus.cc:357] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.416311 18662 raft_consensus.cc:383] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.416555 18662 raft_consensus.cc:738] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.416966 18668 leader_election.cc:290] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:30.417251 18662 consensus_queue.cc:260] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.409569 18661 raft_consensus.cc:738] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.418995 18661 consensus_queue.cc:260] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.419041 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.420135 18662 ts_tablet_manager.cc:1428] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:58:30.421048 18662 tablet_bootstrap.cc:492] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:30.421830 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:30.422870 18469 raft_consensus.cc:2388] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c60c0cbe377c450297e3d536b7f7c89c in current term 1: Already voted for candidate a45f68dd3d9043809f3bb06287f5d2d3 in this term.
I20250112 20:58:30.424459 18585 leader_election.cc:304] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:30.426455 18732 raft_consensus.cc:3054] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.426829 18662 tablet_bootstrap.cc:654] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.438915 18732 raft_consensus.cc:2743] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.445520 18664 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:30.445977 18664 ts_tablet_manager.cc:1397] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.116s user 0.058s sys 0.047s
I20250112 20:58:30.437906 18662 tablet_bootstrap.cc:492] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:30.447748 18662 ts_tablet_manager.cc:1397] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.005s
I20250112 20:58:30.448145 18664 raft_consensus.cc:357] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.448880 18664 raft_consensus.cc:383] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.449146 18664 raft_consensus.cc:738] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.450134 18663 tablet_bootstrap.cc:654] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.449856 18664 consensus_queue.cc:260] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.452646 18661 ts_tablet_manager.cc:1428] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.046s user 0.007s sys 0.000s
I20250112 20:58:30.453563 18661 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:30.456849 18662 raft_consensus.cc:357] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.457499 18662 raft_consensus.cc:383] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.457746 18662 raft_consensus.cc:738] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.463683 18662 consensus_queue.cc:260] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.464190 18664 ts_tablet_manager.cc:1428] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20250112 20:58:30.465155 18664 tablet_bootstrap.cc:492] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:30.465879 18662 ts_tablet_manager.cc:1428] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.018s user 0.011s sys 0.000s
I20250112 20:58:30.466706 18662 tablet_bootstrap.cc:492] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:30.467111 18661 tablet_bootstrap.cc:654] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.475167 18663 tablet_bootstrap.cc:492] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:30.475550 18662 tablet_bootstrap.cc:654] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.476997 18719 raft_consensus.cc:491] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.477737 18664 tablet_bootstrap.cc:654] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.477466 18719 raft_consensus.cc:513] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.475600 18663 ts_tablet_manager.cc:1397] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.065s user 0.011s sys 0.000s
I20250112 20:58:30.479339 18719 leader_election.cc:290] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:30.481781 18663 raft_consensus.cc:357] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.482510 18663 raft_consensus.cc:383] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.482770 18661 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:30.482774 18663 raft_consensus.cc:738] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.483336 18661 ts_tablet_manager.cc:1397] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.000s
I20250112 20:58:30.483644 18663 consensus_queue.cc:260] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.486024 18663 ts_tablet_manager.cc:1428] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.008s user 0.003s sys 0.002s
I20250112 20:58:30.485715 18661 raft_consensus.cc:357] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.486405 18661 raft_consensus.cc:383] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.486668 18661 raft_consensus.cc:738] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.486932 18663 tablet_bootstrap.cc:492] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:30.487243 18661 consensus_queue.cc:260] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.489298 18661 ts_tablet_manager.cc:1428] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250112 20:58:30.490187 18661 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:30.493602 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99e10fcc8944b44aff69f16d268a07c" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:30.498886 18663 tablet_bootstrap.cc:654] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.506421 18662 tablet_bootstrap.cc:492] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:30.506981 18662 ts_tablet_manager.cc:1397] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.040s user 0.012s sys 0.007s
I20250112 20:58:30.510258 18662 raft_consensus.cc:357] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.511155 18662 raft_consensus.cc:383] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.511502 18662 raft_consensus.cc:738] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.506441 18661 tablet_bootstrap.cc:654] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.520737 18661 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:30.521260 18661 ts_tablet_manager.cc:1397] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.031s user 0.009s sys 0.003s
I20250112 20:58:30.522348 18689 consensus_queue.cc:1035] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:30.524237 18661 raft_consensus.cc:357] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.525135 18661 raft_consensus.cc:383] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.525463 18661 raft_consensus.cc:738] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.525730 18732 consensus_queue.cc:1035] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:30.529727 18661 consensus_queue.cc:260] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.530871 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99e10fcc8944b44aff69f16d268a07c" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:30.532845 18361 leader_election.cc:343] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.533995 18661 ts_tablet_manager.cc:1428] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20250112 20:58:30.536912 18661 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:30.538481 18718 raft_consensus.cc:491] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.539113 18718 raft_consensus.cc:513] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.544206 18661 tablet_bootstrap.cc:654] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.549700 18823 raft_consensus.cc:491] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.550364 18823 raft_consensus.cc:513] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.554139 18545 raft_consensus.cc:3054] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:30.561003 18583 consensus_peers.cc:487] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:30.566964 18718 leader_election.cc:290] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:30.567559 18661 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:30.568010 18661 ts_tablet_manager.cc:1397] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.001s
I20250112 20:58:30.569974 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:30.570860 18619 raft_consensus.cc:2463] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:30.571040 18661 raft_consensus.cc:357] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.577207 18661 raft_consensus.cc:383] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.577556 18661 raft_consensus.cc:738] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.578301 18661 consensus_queue.cc:260] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.584779 18661 ts_tablet_manager.cc:1428] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.016s user 0.009s sys 0.000s
I20250112 20:58:30.585879 18661 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:30.586185 18746 consensus_queue.cc:1035] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:30.589809 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.590682 18393 raft_consensus.cc:2463] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:30.592401 18823 leader_election.cc:290] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:30.595372 18435 leader_election.cc:304] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:30.596905 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737c8961a9424363b63a9688853bea1a" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:30.597990 18617 raft_consensus.cc:2463] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:30.599650 18661 tablet_bootstrap.cc:654] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.601018 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737c8961a9424363b63a9688853bea1a" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.609618 18823 raft_consensus.cc:2798] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:30.610289 18823 raft_consensus.cc:491] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:30.610801 18823 raft_consensus.cc:3054] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:30.614885 18362 leader_election.cc:343] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.615358 18362 leader_election.cc:304] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:30.615917 18435 leader_election.cc:304] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
W20250112 20:58:30.617530 18435 leader_election.cc:343] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.618546 18689 raft_consensus.cc:2798] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:30.680811 18689 raft_consensus.cc:491] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:30.681538 18689 raft_consensus.cc:3054] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.619309 18718 raft_consensus.cc:491] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.622174 18662 consensus_queue.cc:260] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.623090 18865 raft_consensus.cc:2743] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.626703 18661 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:30.629843 18812 consensus_queue.cc:1035] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:30.685844 18661 ts_tablet_manager.cc:1397] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.100s user 0.012s sys 0.001s
I20250112 20:58:30.685119 18718 raft_consensus.cc:513] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.688669 18821 raft_consensus.cc:491] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.688897 18718 leader_election.cc:290] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:30.689781 18718 raft_consensus.cc:491] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.662940 18719 consensus_queue.cc:1035] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250112 20:58:30.671933 18436 consensus_peers.cc:487] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:30.689349 18821 raft_consensus.cc:513] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.690927 18661 raft_consensus.cc:357] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.691833 18661 raft_consensus.cc:383] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.692175 18661 raft_consensus.cc:738] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.695286 18821 leader_election.cc:290] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:30.697876 18732 raft_consensus.cc:491] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.698446 18732 raft_consensus.cc:513] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.701614 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.694854 18661 consensus_queue.cc:260] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.702407 18543 raft_consensus.cc:2463] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:30.703507 18661 ts_tablet_manager.cc:1428] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20250112 20:58:30.704063 18732 leader_election.cc:290] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:30.707548 18542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.709375 18542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.711573 18746 raft_consensus.cc:491] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.712042 18746 raft_consensus.cc:513] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.713986 18746 leader_election.cc:290] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:30.714491 18543 raft_consensus.cc:3054] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.715142 18824 raft_consensus.cc:491] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.715631 18824 raft_consensus.cc:513] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.717449 18824 leader_election.cc:290] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:30.661542 18774 consensus_queue.cc:1035] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:30.705264 18584 leader_election.cc:304] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:30.721148 18732 raft_consensus.cc:2798] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:30.721576 18732 raft_consensus.cc:491] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:30.721992 18732 raft_consensus.cc:3054] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:30.734727 18436 leader_election.cc:343] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.735162 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bea0704826246c1b1e2901dbab0cb15" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.744122 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f8bbae0e0c494ab580a5f2338ef4ea" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.744959 18394 raft_consensus.cc:2388] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 1: Already voted for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in this term.
I20250112 20:58:30.750555 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:30.752245 18668 raft_consensus.cc:491] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.762148 18668 raft_consensus.cc:513] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.763341 18823 raft_consensus.cc:513] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.764048 18668 leader_election.cc:290] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:30.765269 18823 leader_election.cc:290] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:30.766014 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759cdb36c54b4950b89e7502ae9e878f" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:30.767868 18584 leader_election.cc:343] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.704324 18661 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:30.735489 18436 leader_election.cc:343] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:30.769412 18436 leader_election.cc:343] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.771298 18663 tablet_bootstrap.cc:492] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:30.771211 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f8bbae0e0c494ab580a5f2338ef4ea" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:30.771759 18663 ts_tablet_manager.cc:1397] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.285s user 0.109s sys 0.121s
I20250112 20:58:30.774256 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759cdb36c54b4950b89e7502ae9e878f" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:30.774315 18663 raft_consensus.cc:357] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.775032 18663 raft_consensus.cc:383] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.775305 18663 raft_consensus.cc:738] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
W20250112 20:58:30.775604 18585 leader_election.cc:343] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:30.776021 18585 leader_election.cc:304] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:30.775895 18663 consensus_queue.cc:260] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.777237 18668 raft_consensus.cc:2743] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.778672 18823 consensus_queue.cc:1035] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:30.781472 18664 tablet_bootstrap.cc:492] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:30.781833 18543 raft_consensus.cc:3054] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:30.783540 18663 ts_tablet_manager.cc:1428] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20250112 20:58:30.784396 18663 tablet_bootstrap.cc:492] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:30.781939 18664 ts_tablet_manager.cc:1397] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.317s user 0.079s sys 0.175s
I20250112 20:58:30.790177 18663 tablet_bootstrap.cc:654] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.791503 18866 raft_consensus.cc:491] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.791954 18866 raft_consensus.cc:513] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:30.793967 18866 leader_election.cc:290] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:30.795871 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:30.806151 18866 consensus_queue.cc:1035] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20250112 20:58:30.801123 18393 raft_consensus.cc:2463] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:30.810870 18664 raft_consensus.cc:357] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.833940 18664 raft_consensus.cc:383] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.834532 18664 raft_consensus.cc:738] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.834214 18718 raft_consensus.cc:513] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:30.829160 18359 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.15.23.190:45335, user_credentials={real_user=slave}} blocked reactor thread for 71752.3us
I20250112 20:58:30.836277 18664 consensus_queue.cc:260] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.851696 18509 leader_election.cc:304] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:30.855510 18812 raft_consensus.cc:2743] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:30.868577 18718 leader_election.cc:290] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:30.871228 18866 raft_consensus.cc:491] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.871908 18866 raft_consensus.cc:513] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.876963 18662 ts_tablet_manager.cc:1428] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.370s user 0.035s sys 0.071s
I20250112 20:58:30.877904 18662 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:30.881007 18866 leader_election.cc:290] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:30.890308 18662 tablet_bootstrap.cc:654] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.891078 18664 ts_tablet_manager.cc:1428] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.101s user 0.008s sys 0.011s
I20250112 20:58:30.892113 18664 tablet_bootstrap.cc:492] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:30.896034 18689 raft_consensus.cc:513] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:30.898420 18689 leader_election.cc:290] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:30.900452 18664 tablet_bootstrap.cc:654] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.913045 18662 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:30.913520 18662 ts_tablet_manager.cc:1397] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.004s
I20250112 20:58:30.915498 18664 tablet_bootstrap.cc:492] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:30.918529 18664 ts_tablet_manager.cc:1397] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.027s user 0.014s sys 0.000s
I20250112 20:58:30.921720 18662 raft_consensus.cc:357] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.922415 18662 raft_consensus.cc:383] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.922659 18662 raft_consensus.cc:738] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.925477 18664 raft_consensus.cc:357] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.927779 18664 raft_consensus.cc:383] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:30.926288 18662 consensus_queue.cc:260] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.928833 18664 raft_consensus.cc:738] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:30.930789 18662 ts_tablet_manager.cc:1428] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20250112 20:58:30.931718 18662 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:30.930446 18664 consensus_queue.cc:260] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:30.935389 18664 ts_tablet_manager.cc:1428] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20250112 20:58:30.938495 18664 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:30.955521 18664 tablet_bootstrap.cc:654] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.959174 18662 tablet_bootstrap.cc:654] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:30.970284 18774 raft_consensus.cc:491] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:30.970985 18774 raft_consensus.cc:513] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:30.974733 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:30.978861 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.979772 18541 raft_consensus.cc:2463] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:30.987459 18361 leader_election.cc:304] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:30.987664 18542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:30.989689 18719 raft_consensus.cc:2798] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:30.990093 18719 raft_consensus.cc:491] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:30.990475 18719 raft_consensus.cc:3054] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:30.999105 18361 leader_election.cc:343] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.002152 18865 raft_consensus.cc:491] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.002871 18865 raft_consensus.cc:513] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.011071 18774 leader_election.cc:290] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
W20250112 20:58:31.011695 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 394195us
I20250112 20:58:31.014309 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bea0704826246c1b1e2901dbab0cb15" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:31.014954 18617 raft_consensus.cc:2463] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:31.015343 18661 tablet_bootstrap.cc:654] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.017095 18718 raft_consensus.cc:491] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.016029 18689 raft_consensus.cc:491] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.017678 18865 leader_election.cc:290] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:31.030107 18663 tablet_bootstrap.cc:492] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:31.030725 18663 ts_tablet_manager.cc:1397] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.247s user 0.068s sys 0.072s
I20250112 20:58:31.024396 18689 raft_consensus.cc:513] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.035938 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.036574 18617 raft_consensus.cc:3054] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.028877 18774 consensus_queue.cc:1035] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20250112 20:58:31.040555 18812 raft_consensus.cc:491] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.024657 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:31.041334 18362 leader_election.cc:343] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.047689 18362 leader_election.cc:304] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.041190 18812 raft_consensus.cc:513] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.041442 18435 leader_election.cc:304] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.049026 18870 raft_consensus.cc:2798] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.049377 18870 raft_consensus.cc:491] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.049654 18870 raft_consensus.cc:3054] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.022293 18718 raft_consensus.cc:513] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.054193 18718 leader_election.cc:290] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.055858 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737c8961a9424363b63a9688853bea1a" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.056034 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8300b885639841769a76417fb45f2db7" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:31.057096 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:31.057621 18436 leader_election.cc:343] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.058230 18619 raft_consensus.cc:2463] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:31.059064 18812 leader_election.cc:290] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.056496 18618 raft_consensus.cc:3054] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.059666 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8300b885639841769a76417fb45f2db7" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:31.059732 18719 raft_consensus.cc:513] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.061909 18774 raft_consensus.cc:2743] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:31.065515 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:31.066115 18470 raft_consensus.cc:2463] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:31.070472 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:31.071079 18471 raft_consensus.cc:2463] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:31.072387 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:31.073048 18468 raft_consensus.cc:2463] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:31.076736 18719 leader_election.cc:290] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.078327 18511 leader_election.cc:304] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:31.079366 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.079958 18393 raft_consensus.cc:2463] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:31.082899 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:31.083549 18392 raft_consensus.cc:3054] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.085011 18618 raft_consensus.cc:2463] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:31.087210 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.087797 18395 raft_consensus.cc:2463] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:31.056142 18870 raft_consensus.cc:513] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.088661 18812 raft_consensus.cc:2798] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.094552 18362 leader_election.cc:304] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:31.095515 18662 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:31.096148 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:31.096238 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737c8961a9424363b63a9688853bea1a" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:31.096944 18812 raft_consensus.cc:491] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.103101 18812 raft_consensus.cc:3054] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.103955 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bea0704826246c1b1e2901dbab0cb15" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:31.104542 18543 raft_consensus.cc:3054] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.105796 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:31.106400 18615 raft_consensus.cc:2463] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:31.107486 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:31.108028 18545 raft_consensus.cc:3054] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:31.108301 18361 leader_election.cc:343] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.110482 18362 leader_election.cc:304] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:31.111186 18774 raft_consensus.cc:2798] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.093868 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:31.111534 18774 raft_consensus.cc:491] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.111896 18774 raft_consensus.cc:3054] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.114342 18617 raft_consensus.cc:2463] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
W20250112 20:58:31.089046 18435 leader_election.cc:343] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.107314 18663 raft_consensus.cc:357] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.117511 18663 raft_consensus.cc:383] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.117787 18663 raft_consensus.cc:738] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.118279 18663 consensus_queue.cc:260] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.089636 18732 raft_consensus.cc:513] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.119365 18868 raft_consensus.cc:2798] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.119731 18868 raft_consensus.cc:491] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.119915 18616 raft_consensus.cc:2388] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a45f68dd3d9043809f3bb06287f5d2d3 in current term 1: Already voted for candidate c60c0cbe377c450297e3d536b7f7c89c in this term.
I20250112 20:58:31.120029 18868 raft_consensus.cc:3054] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.122975 18663 ts_tablet_manager.cc:1428] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.092s user 0.004s sys 0.003s
I20250112 20:58:31.123840 18663 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:31.099222 18662 ts_tablet_manager.cc:1397] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.168s user 0.016s sys 0.009s
I20250112 20:58:31.127948 18614 raft_consensus.cc:2463] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:31.129371 18689 leader_election.cc:290] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
W20250112 20:58:31.126724 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 44693.9us
I20250112 20:58:31.130679 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99e10fcc8944b44aff69f16d268a07c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:31.131215 18663 tablet_bootstrap.cc:654] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.130743 18435 leader_election.cc:304] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.132795 18718 raft_consensus.cc:2798] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.133141 18718 raft_consensus.cc:491] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.133459 18718 raft_consensus.cc:3054] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.133648 18545 raft_consensus.cc:2463] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:31.134577 18868 raft_consensus.cc:513] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.135252 18361 leader_election.cc:304] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:31.138123 18435 leader_election.cc:304] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:31.138864 18821 raft_consensus.cc:2798] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.139310 18821 raft_consensus.cc:695] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:31.140081 18821 consensus_queue.cc:237] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.141820 18812 raft_consensus.cc:513] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.142411 18866 raft_consensus.cc:2798] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.142903 18866 raft_consensus.cc:695] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
W20250112 20:58:31.143949 18436 leader_election.cc:343] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.143605 18866 consensus_queue.cc:237] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.144979 18812 leader_election.cc:290] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.146409 18732 leader_election.cc:290] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:31.145717 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:31.147428 18544 raft_consensus.cc:2388] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c60c0cbe377c450297e3d536b7f7c89c in current term 1: Already voted for candidate a45f68dd3d9043809f3bb06287f5d2d3 in this term.
I20250112 20:58:31.149080 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:31.149960 18435 leader_election.cc:304] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:31.150359 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.150676 18689 raft_consensus.cc:2798] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.150614 18662 raft_consensus.cc:357] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.151125 18614 raft_consensus.cc:3054] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.151347 18662 raft_consensus.cc:383] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.151335 18689 raft_consensus.cc:695] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:31.152122 18435 leader_election.cc:304] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.153221 18662 raft_consensus.cc:738] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.153509 18823 raft_consensus.cc:2798] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.153843 18823 raft_consensus.cc:491] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.154270 18823 raft_consensus.cc:3054] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.153877 18662 consensus_queue.cc:260] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.156255 18662 ts_tablet_manager.cc:1428] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20250112 20:58:31.157142 18662 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:31.152827 18870 leader_election.cc:290] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.152369 18689 consensus_queue.cc:237] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.124066 18774 raft_consensus.cc:513] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.161548 18664 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:31.164028 18392 raft_consensus.cc:2463] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:31.164047 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:31.164366 18664 ts_tablet_manager.cc:1397] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.226s user 0.010s sys 0.010s
I20250112 20:58:31.164755 18471 raft_consensus.cc:3054] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.164836 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
W20250112 20:58:31.166033 18361 leader_election.cc:343] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.111860 18543 raft_consensus.cc:2463] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:31.167649 18774 leader_election.cc:290] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:31.172333 18471 raft_consensus.cc:2463] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:31.180727 18664 raft_consensus.cc:357] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.181875 18664 raft_consensus.cc:383] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.182271 18664 raft_consensus.cc:738] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.183147 18664 consensus_queue.cc:260] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.192230 18664 ts_tablet_manager.cc:1428] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20250112 20:58:31.192911 18664 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:31.204773 18436 leader_election.cc:304] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:31.205596 18661 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:31.206244 18661 ts_tablet_manager.cc:1397] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.502s user 0.020s sys 0.000s
I20250112 20:58:31.208969 18824 raft_consensus.cc:2798] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.209146 18868 leader_election.cc:290] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:31.212471 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bea0704826246c1b1e2901dbab0cb15" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.213129 18616 raft_consensus.cc:3054] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.216673 18661 raft_consensus.cc:357] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.200054 18362 leader_election.cc:304] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.217409 18661 raft_consensus.cc:383] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.217691 18661 raft_consensus.cc:738] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.218612 18774 raft_consensus.cc:2798] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.219075 18774 raft_consensus.cc:695] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:31.219022 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:31.219612 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.219823 18469 raft_consensus.cc:3054] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.220431 18618 raft_consensus.cc:2388] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a45f68dd3d9043809f3bb06287f5d2d3 in current term 1: Already voted for candidate c60c0cbe377c450297e3d536b7f7c89c in this term.
I20250112 20:58:31.220044 18774 consensus_queue.cc:237] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.221669 18662 tablet_bootstrap.cc:654] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.218242 18661 consensus_queue.cc:260] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.147958 18718 raft_consensus.cc:513] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.230592 18718 leader_election.cc:290] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.234642 18663 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:31.236652 18663 ts_tablet_manager.cc:1397] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.113s user 0.034s sys 0.039s
I20250112 20:58:31.241007 18661 ts_tablet_manager.cc:1428] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.034s user 0.007s sys 0.000s
I20250112 20:58:31.243145 18661 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:31.249946 18664 tablet_bootstrap.cc:654] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.258265 18468 raft_consensus.cc:2388] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 1: Already voted for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in this term.
I20250112 20:58:31.261781 18661 tablet_bootstrap.cc:654] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
W20250112 20:58:31.276499 18360 leader_election.cc:343] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.276672 18823 raft_consensus.cc:513] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.269269 18824 raft_consensus.cc:695] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:31.285130 18823 leader_election.cc:290] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.285683 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:31.285746 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8300b885639841769a76417fb45f2db7" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
W20250112 20:58:31.300359 18436 leader_election.cc:343] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.270864 18435 leader_election.cc:304] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.305835 18823 raft_consensus.cc:2758] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20250112 20:58:31.306962 18823 consensus_queue.cc:1035] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:31.307055 18435 leader_election.cc:304] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
W20250112 20:58:31.309840 18435 leader_election.cc:343] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.311036 18469 raft_consensus.cc:2463] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:31.311559 18689 raft_consensus.cc:2758] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20250112 20:58:31.314980 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99e10fcc8944b44aff69f16d268a07c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.315846 18395 raft_consensus.cc:2463] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:31.316995 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.318526 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8300b885639841769a76417fb45f2db7" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.319293 18617 raft_consensus.cc:3054] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.286420 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9028573e647bba81035b9c4a15146" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:31.321772 18543 raft_consensus.cc:2388] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2f9ce9a0415c4546b9a764ac06cefb38 in current term 1: Already voted for candidate 88ef62b247304e079788e24ddfdaa4fb in this term.
I20250112 20:58:31.325685 18362 leader_election.cc:304] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:31.326797 18868 raft_consensus.cc:2798] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.327268 18868 raft_consensus.cc:695] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:31.328078 18868 consensus_queue.cc:237] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.330981 18614 raft_consensus.cc:2463] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:31.332141 18509 leader_election.cc:304] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38
I20250112 20:58:31.334908 18880 raft_consensus.cc:2798] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.335497 18880 raft_consensus.cc:695] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:31.336463 18880 consensus_queue.cc:237] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.347745 18435 leader_election.cc:304] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.348996 18435 leader_election.cc:304] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:31.350180 18870 raft_consensus.cc:2798] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.350543 18870 raft_consensus.cc:491] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.350849 18870 raft_consensus.cc:3054] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.355548 18290 catalog_manager.cc:5526] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:31.362133 18893 raft_consensus.cc:2743] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250112 20:58:31.362470 18663 raft_consensus.cc:357] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.363162 18663 raft_consensus.cc:383] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.363425 18663 raft_consensus.cc:738] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.367691 18688 raft_consensus.cc:491] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.367882 18870 raft_consensus.cc:513] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.368371 18688 raft_consensus.cc:513] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.381968 18688 leader_election.cc:290] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:31.382275 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99e10fcc8944b44aff69f16d268a07c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:31.382889 18395 raft_consensus.cc:3054] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.361635 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.369562 18617 raft_consensus.cc:2463] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:31.384258 18865 raft_consensus.cc:491] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.385252 18435 leader_election.cc:304] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.384953 18865 raft_consensus.cc:513] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.385725 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99e10fcc8944b44aff69f16d268a07c" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:31.386063 18718 raft_consensus.cc:2798] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.386569 18718 raft_consensus.cc:695] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:31.386696 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:31.387136 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.352840 18824 consensus_queue.cc:237] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.387390 18718 consensus_queue.cc:237] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:31.388196 18360 leader_election.cc:343] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.390463 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:31.379645 18616 raft_consensus.cc:2463] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:31.379348 18663 consensus_queue.cc:260] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.370573 18583 leader_election.cc:304] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:31.399461 18663 ts_tablet_manager.cc:1428] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.161s user 0.013s sys 0.000s
I20250112 20:58:31.400211 18663 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
W20250112 20:58:31.343194 18342 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:58:31.382301 18870 leader_election.cc:290] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:31.421684 18664 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:31.425088 18664 ts_tablet_manager.cc:1397] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.232s user 0.018s sys 0.001s
I20250112 20:58:31.415742 18736 raft_consensus.cc:2743] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20250112 20:58:31.417757 18361 leader_election.cc:343] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:31.409729 18436 leader_election.cc:343] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.422708 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:31.427579 18664 raft_consensus.cc:357] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:31.416273 18511 leader_election.cc:343] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.417239 18865 leader_election.cc:290] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.429131 18664 raft_consensus.cc:383] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.432029 18664 raft_consensus.cc:738] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.432919 18664 consensus_queue.cc:260] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:31.418392 18583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 47911.8us
I20250112 20:58:31.441502 18664 ts_tablet_manager.cc:1428] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.016s user 0.007s sys 0.001s
I20250112 20:58:31.442689 18664 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
W20250112 20:58:31.435777 18509 leader_election.cc:343] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.435631 18616 raft_consensus.cc:2463] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:31.453478 18661 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:31.453984 18661 ts_tablet_manager.cc:1397] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.211s user 0.031s sys 0.073s
I20250112 20:58:31.456523 18661 raft_consensus.cc:357] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.457254 18661 raft_consensus.cc:383] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.457533 18661 raft_consensus.cc:738] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.457785 18395 raft_consensus.cc:2463] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:31.458407 18662 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:31.458127 18661 consensus_queue.cc:260] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.458966 18662 ts_tablet_manager.cc:1397] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.302s user 0.027s sys 0.033s
I20250112 20:58:31.460048 18661 ts_tablet_manager.cc:1428] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:58:31.461185 18661 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:31.461220 18360 leader_election.cc:304] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.462193 18719 raft_consensus.cc:2798] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.462556 18719 raft_consensus.cc:491] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.462870 18719 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.474020 18661 tablet_bootstrap.cc:654] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.477001 18662 raft_consensus.cc:357] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.488816 18663 tablet_bootstrap.cc:654] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.489032 18662 raft_consensus.cc:383] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.489508 18662 raft_consensus.cc:738] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.477655 18664 tablet_bootstrap.cc:654] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.479506 18509 leader_election.cc:304] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:31.490303 18662 consensus_queue.cc:260] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:31.501609 18509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 83869.6us
I20250112 20:58:31.490289 18435 leader_election.cc:304] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.504855 18662 ts_tablet_manager.cc:1428] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.046s user 0.008s sys 0.000s
I20250112 20:58:31.506079 18662 tablet_bootstrap.cc:492] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:31.506211 18893 raft_consensus.cc:2798] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.507781 18893 raft_consensus.cc:695] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:31.508584 18893 consensus_queue.cc:237] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.509918 18688 raft_consensus.cc:2743] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:31.519431 18719 raft_consensus.cc:513] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.521631 18719 leader_election.cc:290] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.551843 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
W20250112 20:58:31.553601 18435 consensus_peers.cc:487] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:31.554208 18662 tablet_bootstrap.cc:654] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.559347 18661 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:31.559255 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.559988 18617 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.559924 18661 ts_tablet_manager.cc:1397] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.099s user 0.004s sys 0.016s
I20250112 20:58:31.564157 18661 raft_consensus.cc:357] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.564831 18661 raft_consensus.cc:383] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.565074 18661 raft_consensus.cc:738] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
W20250112 20:58:31.566223 18361 leader_election.cc:343] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.565706 18661 consensus_queue.cc:260] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.568396 18661 ts_tablet_manager.cc:1428] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:58:31.569262 18661 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:31.571065 18870 consensus_queue.cc:1035] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:31.574985 18661 tablet_bootstrap.cc:654] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.577735 18664 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:31.577864 18663 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:31.578202 18664 ts_tablet_manager.cc:1397] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.136s user 0.009s sys 0.004s
I20250112 20:58:31.578437 18663 ts_tablet_manager.cc:1397] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.178s user 0.012s sys 0.019s
I20250112 20:58:31.580837 18617 raft_consensus.cc:2463] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:31.582235 18360 leader_election.cc:304] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.583351 18865 raft_consensus.cc:2798] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.583808 18865 raft_consensus.cc:695] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:31.585218 18663 raft_consensus.cc:357] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.584553 18865 consensus_queue.cc:237] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.586344 18663 raft_consensus.cc:383] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.586684 18663 raft_consensus.cc:738] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.587420 18663 consensus_queue.cc:260] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.590703 18664 raft_consensus.cc:357] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.593364 18664 raft_consensus.cc:383] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.593569 18664 raft_consensus.cc:738] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.594048 18664 consensus_queue.cc:260] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.596119 18663 ts_tablet_manager.cc:1428] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20250112 20:58:31.596863 18663 tablet_bootstrap.cc:492] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:31.603837 18664 ts_tablet_manager.cc:1428] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.025s user 0.004s sys 0.002s
I20250112 20:58:31.604640 18664 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:31.612191 18812 raft_consensus.cc:491] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.612859 18812 raft_consensus.cc:513] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.615254 18812 leader_election.cc:290] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:31.617218 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99e10fcc8944b44aff69f16d268a07c" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:31.617502 18894 raft_consensus.cc:491] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.618515 18294 catalog_manager.cc:5526] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:31.628268 18894 raft_consensus.cc:513] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.634099 18719 raft_consensus.cc:491] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.634593 18719 raft_consensus.cc:513] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.635113 18894 leader_election.cc:290] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:31.635881 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c99e10fcc8944b44aff69f16d268a07c" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.636369 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:31.636720 18394 raft_consensus.cc:2388] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 1: Already voted for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in this term.
I20250112 20:58:31.637620 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:31.638293 18470 raft_consensus.cc:2463] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:31.639087 18509 leader_election.cc:304] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:31.639894 18812 raft_consensus.cc:3054] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.642114 18362 leader_election.cc:304] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:31.643152 18774 raft_consensus.cc:2798] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.643548 18774 raft_consensus.cc:491] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.643872 18774 raft_consensus.cc:3054] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.646600 18812 raft_consensus.cc:2743] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:31.647593 18663 tablet_bootstrap.cc:654] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
W20250112 20:58:31.637663 18436 leader_election.cc:343] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.654791 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:31.655443 18614 raft_consensus.cc:2463] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:31.656656 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.657402 18394 raft_consensus.cc:2463] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:31.662243 18664 tablet_bootstrap.cc:654] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.662343 18662 tablet_bootstrap.cc:492] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:31.662879 18662 ts_tablet_manager.cc:1397] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.157s user 0.017s sys 0.001s
I20250112 20:58:31.663105 18435 leader_election.cc:304] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:31.666997 18893 raft_consensus.cc:2798] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:31.667375 18893 raft_consensus.cc:491] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:31.667721 18893 raft_consensus.cc:3054] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.660763 18719 leader_election.cc:290] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:31.676513 18291 catalog_manager.cc:5526] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:31.677623 18894 consensus_queue.cc:1035] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:31.677500 18893 raft_consensus.cc:513] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.678414 18291 catalog_manager.cc:5526] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:31.682708 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:31.683430 18394 raft_consensus.cc:3054] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.683858 18774 raft_consensus.cc:513] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.687744 18661 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:31.687958 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:31.688324 18661 ts_tablet_manager.cc:1397] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.119s user 0.028s sys 0.033s
I20250112 20:58:31.688573 18470 raft_consensus.cc:3054] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.693660 18880 raft_consensus.cc:491] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.694409 18880 raft_consensus.cc:513] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.695169 18470 raft_consensus.cc:2463] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:31.696195 18362 leader_election.cc:304] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:31.696971 18894 consensus_queue.cc:1035] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:31.697198 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f0a2bc717d404f8f3afd03c3d5a60c" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.696990 18719 raft_consensus.cc:2798] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.698659 18719 raft_consensus.cc:695] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:31.699097 18662 raft_consensus.cc:357] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.699797 18662 raft_consensus.cc:383] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.700129 18662 raft_consensus.cc:738] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.701495 18880 leader_election.cc:290] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:31.703023 18732 raft_consensus.cc:491] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.703215 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f0a2bc717d404f8f3afd03c3d5a60c" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:31.703500 18732 raft_consensus.cc:513] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.703706 18661 raft_consensus.cc:357] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.704346 18661 raft_consensus.cc:383] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.704648 18661 raft_consensus.cc:738] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.703979 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:31.704946 18511 leader_election.cc:304] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:31.699425 18719 consensus_queue.cc:237] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.705180 18617 raft_consensus.cc:3054] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.706383 18662 consensus_queue.cc:260] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.712303 18732 leader_election.cc:290] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:31.708930 18893 leader_election.cc:290] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:31.709282 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af0c89ea5b0d4655959ccaad0b838bf4" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:31.709904 18812 raft_consensus.cc:2743] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:31.715924 18662 ts_tablet_manager.cc:1428] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.053s user 0.005s sys 0.002s
I20250112 20:58:31.680100 18291 catalog_manager.cc:5526] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:31.709362 18661 consensus_queue.cc:260] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.710666 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:31.717911 18291 catalog_manager.cc:5526] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:31.699657 18394 raft_consensus.cc:2463] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
W20250112 20:58:31.719014 18436 leader_election.cc:343] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.708303 18774 leader_election.cc:290] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:31.721378 18661 ts_tablet_manager.cc:1428] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20250112 20:58:31.714721 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af0c89ea5b0d4655959ccaad0b838bf4" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
W20250112 20:58:31.723006 18584 leader_election.cc:343] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.721012 18435 leader_election.cc:304] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
W20250112 20:58:31.723768 18583 leader_election.cc:343] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.724224 18583 leader_election.cc:304] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:31.726337 18732 raft_consensus.cc:2743] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:31.724442 18894 raft_consensus.cc:2798] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:31.725167 18663 tablet_bootstrap.cc:492] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
W20250112 20:58:31.725324 18361 consensus_peers.cc:487] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:31.727649 18663 ts_tablet_manager.cc:1397] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.131s user 0.014s sys 0.010s
I20250112 20:58:31.728137 18894 raft_consensus.cc:695] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:31.731365 18662 tablet_bootstrap.cc:492] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:31.733256 18661 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:31.735340 18663 raft_consensus.cc:357] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.737951 18663 raft_consensus.cc:383] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.738150 18663 raft_consensus.cc:738] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.738690 18663 consensus_queue.cc:260] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:31.740156 18663 ts_tablet_manager.cc:1428] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20250112 20:58:31.740823 18663 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:31.728576 18617 raft_consensus.cc:2463] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:31.747454 18661 tablet_bootstrap.cc:654] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.751016 18732 raft_consensus.cc:491] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.729100 18894 consensus_queue.cc:237] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:31.752316 18732 raft_consensus.cc:513] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.765364 18663 tablet_bootstrap.cc:654] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.766144 18662 tablet_bootstrap.cc:654] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.758268 18543 raft_consensus.cc:3054] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.763654 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f52ac1efd0e417393e5127713a43a18" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.764855 18732 leader_election.cc:290] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:31.765605 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f52ac1efd0e417393e5127713a43a18" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:31.781466 18583 leader_election.cc:343] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.783630 18664 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:31.789821 18664 ts_tablet_manager.cc:1397] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.185s user 0.023s sys 0.027s
I20250112 20:58:31.790767 18880 consensus_queue.cc:1035] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:31.800015 18664 raft_consensus.cc:357] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.800516 18664 raft_consensus.cc:383] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.800859 18664 raft_consensus.cc:738] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.801550 18664 consensus_queue.cc:260] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.810289 18664 ts_tablet_manager.cc:1428] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
I20250112 20:58:31.807703 18719 consensus_queue.cc:1035] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20250112 20:58:31.823196 18584 leader_election.cc:343] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.823731 18584 leader_election.cc:304] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:31.828828 18736 raft_consensus.cc:2743] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:31.832791 18719 consensus_queue.cc:1035] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:58:31.833678 18360 consensus_peers.cc:487] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 -> Peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Couldn't send request to peer c60c0cbe377c450297e3d536b7f7c89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:31.837086 18774 consensus_queue.cc:1035] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:31.837961 18290 catalog_manager.cc:5526] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:31.839208 18774 consensus_queue.cc:1035] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:31.839581 18290 catalog_manager.cc:5526] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:31.841437 18290 catalog_manager.cc:5526] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:31.853906 18664 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:31.865859 18664 tablet_bootstrap.cc:654] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.901721 18812 raft_consensus.cc:491] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.902333 18812 raft_consensus.cc:513] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.906066 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:31.914090 18661 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:31.914585 18661 ts_tablet_manager.cc:1397] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.182s user 0.035s sys 0.025s
I20250112 20:58:31.915443 18294 catalog_manager.cc:5526] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:31.916476 18894 consensus_queue.cc:1035] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:31.904582 18812 leader_election.cc:290] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:31.923439 18661 raft_consensus.cc:357] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.924307 18661 raft_consensus.cc:383] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.924667 18661 raft_consensus.cc:738] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.925379 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:31.926152 18614 raft_consensus.cc:2388] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 1: Already voted for candidate a45f68dd3d9043809f3bb06287f5d2d3 in this term.
I20250112 20:58:31.925467 18661 consensus_queue.cc:260] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.928542 18661 ts_tablet_manager.cc:1428] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20250112 20:58:31.931830 18661 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:31.935377 18509 leader_election.cc:304] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:31.936875 18688 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:31.937502 18661 tablet_bootstrap.cc:654] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:31.953250 18663 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:31.953653 18663 ts_tablet_manager.cc:1397] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.213s user 0.010s sys 0.016s
I20250112 20:58:31.957412 18662 tablet_bootstrap.cc:492] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:31.957867 18662 ts_tablet_manager.cc:1397] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.227s user 0.010s sys 0.008s
I20250112 20:58:31.955513 18866 raft_consensus.cc:491] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.967967 18866 raft_consensus.cc:513] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:31.964027 18774 raft_consensus.cc:491] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.967443 18688 raft_consensus.cc:2743] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:31.980270 18663 raft_consensus.cc:357] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.981477 18663 raft_consensus.cc:383] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:31.981803 18663 raft_consensus.cc:738] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:31.982658 18663 consensus_queue.cc:260] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.984205 18868 raft_consensus.cc:491] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:31.984671 18868 raft_consensus.cc:513] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:31.986485 18868 leader_election.cc:290] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:31.988747 18866 leader_election.cc:290] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:31.989724 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d5dc78c20741968c7ca8f7ecec1c07" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:31.990139 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737c8961a9424363b63a9688853bea1a" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:31.990929 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d5dc78c20741968c7ca8f7ecec1c07" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:31.992269 18361 leader_election.cc:343] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.993306 18824 consensus_queue.cc:1035] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:31.994899 18663 ts_tablet_manager.cc:1428] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.041s user 0.005s sys 0.003s
I20250112 20:58:31.996258 18664 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:31.996969 18664 ts_tablet_manager.cc:1397] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.143s user 0.031s sys 0.009s
I20250112 20:58:31.998245 18663 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:31.994169 18865 consensus_queue.cc:1035] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20250112 20:58:32.005707 18662 raft_consensus.cc:357] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.006158 18662 raft_consensus.cc:383] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.006309 18662 raft_consensus.cc:738] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.007793 18662 consensus_queue.cc:260] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:32.003679 18360 leader_election.cc:343] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:31.995836 18774 raft_consensus.cc:513] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.014391 18360 leader_election.cc:304] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.021607 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:32.021958 18774 leader_election.cc:290] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:32.015571 18894 raft_consensus.cc:491] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250112 20:58:32.022882 18362 leader_election.cc:343] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.023106 18894 raft_consensus.cc:513] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.034268 18894 leader_election.cc:290] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:32.018884 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "737c8961a9424363b63a9688853bea1a" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.021735 18866 raft_consensus.cc:2743] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.013484 18664 raft_consensus.cc:357] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.035946 18664 raft_consensus.cc:383] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.036253 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe90092ab277453fb41e05a8be798304" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.016649 18661 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:32.018605 18893 consensus_queue.cc:1035] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:32.036530 18663 tablet_bootstrap.cc:654] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.039700 18662 ts_tablet_manager.cc:1428] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.082s user 0.003s sys 0.002s
I20250112 20:58:32.036252 18664 raft_consensus.cc:738] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.056113 18661 ts_tablet_manager.cc:1397] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.124s user 0.019s sys 0.021s
I20250112 20:58:32.056679 18662 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:32.056612 18664 consensus_queue.cc:260] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.036926 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:32.059691 18361 leader_election.cc:343] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.060169 18361 leader_election.cc:304] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:32.059069 18664 ts_tablet_manager.cc:1428] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.062s user 0.006s sys 0.008s
I20250112 20:58:32.061453 18664 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:32.062037 18866 raft_consensus.cc:2743] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.067373 18661 raft_consensus.cc:357] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.067896 18661 raft_consensus.cc:383] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.067953 18664 tablet_bootstrap.cc:654] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.068082 18661 raft_consensus.cc:738] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.068714 18661 consensus_queue.cc:260] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.069775 18661 ts_tablet_manager.cc:1428] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20250112 20:58:32.070348 18661 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:32.040521 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe90092ab277453fb41e05a8be798304" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:32.076371 18435 leader_election.cc:343] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.038218 18291 catalog_manager.cc:5526] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:32.052874 18360 leader_election.cc:304] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.084695 18893 raft_consensus.cc:491] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.087743 18291 catalog_manager.cc:5526] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:32.089876 18736 raft_consensus.cc:491] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.090359 18736 raft_consensus.cc:513] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.092018 18736 leader_election.cc:290] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
W20250112 20:58:32.077764 18436 consensus_peers.cc:487] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:32.092398 18893 raft_consensus.cc:513] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:32.094152 18436 leader_election.cc:343] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.094084 18718 consensus_queue.cc:1035] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:32.094651 18436 leader_election.cc:304] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.095088 18893 leader_election.cc:290] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:32.095697 18893 raft_consensus.cc:2743] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.063232 18940 consensus_queue.cc:1035] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20250112 20:58:32.097393 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:32.102991 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759cdb36c54b4950b89e7502ae9e878f" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.103566 18616 raft_consensus.cc:2463] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:32.106482 18435 leader_election.cc:304] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:32.105322 18866 raft_consensus.cc:2743] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.116111 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:32.105944 18584 leader_election.cc:343] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.112769 18824 raft_consensus.cc:2798] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:32.118853 18824 raft_consensus.cc:491] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:32.119221 18824 raft_consensus.cc:3054] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.124069 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759cdb36c54b4950b89e7502ae9e878f" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:32.125568 18866 consensus_queue.cc:1035] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:58:32.113421 18360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 73403.8us
I20250112 20:58:32.145177 18824 raft_consensus.cc:513] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.147423 18392 raft_consensus.cc:3054] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.148749 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759cdb36c54b4950b89e7502ae9e878f" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:32.149542 18618 raft_consensus.cc:3054] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.150588 18661 tablet_bootstrap.cc:654] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
W20250112 20:58:32.157071 18585 leader_election.cc:343] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.157720 18585 leader_election.cc:304] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:32.158663 18736 raft_consensus.cc:2743] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.165234 18664 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:32.165715 18664 ts_tablet_manager.cc:1397] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.104s user 0.008s sys 0.011s
W20250112 20:58:32.166929 18436 leader_election.cc:343] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.170118 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759cdb36c54b4950b89e7502ae9e878f" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:32.168123 18664 raft_consensus.cc:357] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.171731 18664 raft_consensus.cc:383] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.172014 18664 raft_consensus.cc:738] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.172650 18664 consensus_queue.cc:260] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.174975 18894 consensus_queue.cc:1035] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:32.187470 18664 ts_tablet_manager.cc:1428] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20250112 20:58:32.178037 18868 consensus_queue.cc:1035] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20250112 20:58:32.185559 18824 leader_election.cc:290] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:32.192659 18662 tablet_bootstrap.cc:654] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.193197 18664 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:32.201545 18541 raft_consensus.cc:3054] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.200256 18774 raft_consensus.cc:491] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.205456 18774 raft_consensus.cc:513] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.207859 18774 leader_election.cc:290] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:32.208940 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:32.210646 18362 leader_election.cc:343] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.215021 18618 raft_consensus.cc:2463] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:32.216878 18435 leader_election.cc:304] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:32.222258 18824 raft_consensus.cc:2798] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:32.222718 18824 raft_consensus.cc:695] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:32.223479 18824 consensus_queue.cc:237] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.225081 18663 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:32.225582 18663 ts_tablet_manager.cc:1397] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.230s user 0.017s sys 0.006s
I20250112 20:58:32.227916 18663 raft_consensus.cc:357] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.231426 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.233728 18663 raft_consensus.cc:383] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.234078 18663 raft_consensus.cc:738] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.234772 18663 consensus_queue.cc:260] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.236419 18663 ts_tablet_manager.cc:1428] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20250112 20:58:32.241730 18663 tablet_bootstrap.cc:492] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
W20250112 20:58:32.251288 18436 leader_election.cc:343] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:32.273725 18360 leader_election.cc:343] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.274186 18360 leader_election.cc:304] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.276572 18661 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:32.277082 18661 ts_tablet_manager.cc:1397] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.207s user 0.018s sys 0.019s
I20250112 20:58:32.282553 18865 raft_consensus.cc:2743] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.296216 18662 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:32.296730 18662 ts_tablet_manager.cc:1397] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.240s user 0.016s sys 0.004s
I20250112 20:58:32.299563 18661 raft_consensus.cc:357] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.300619 18661 raft_consensus.cc:383] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.300984 18661 raft_consensus.cc:738] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.300987 18662 raft_consensus.cc:357] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.301828 18662 raft_consensus.cc:383] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.302099 18662 raft_consensus.cc:738] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.302220 18661 consensus_queue.cc:260] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:32.306804 18436 consensus_peers.cc:487] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:32.302685 18662 consensus_queue.cc:260] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.315613 18662 ts_tablet_manager.cc:1428] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.019s user 0.004s sys 0.000s
I20250112 20:58:32.318292 18661 ts_tablet_manager.cc:1428] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
I20250112 20:58:32.319175 18661 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:32.324774 18824 raft_consensus.cc:491] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.327078 18824 raft_consensus.cc:513] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.335060 18774 consensus_queue.cc:1035] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:32.336681 18664 tablet_bootstrap.cc:654] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.339867 18663 tablet_bootstrap.cc:654] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.316542 18662 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:32.350140 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:32.351752 18436 leader_election.cc:343] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.316449 18293 catalog_manager.cc:5526] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:32.355516 18824 leader_election.cc:290] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:32.364704 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:32.365425 18395 raft_consensus.cc:2463] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:32.348347 18661 tablet_bootstrap.cc:654] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.366634 18664 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:32.367204 18664 ts_tablet_manager.cc:1397] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.175s user 0.013s sys 0.000s
I20250112 20:58:32.371193 18663 tablet_bootstrap.cc:492] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:32.371587 18774 raft_consensus.cc:491] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.371686 18663 ts_tablet_manager.cc:1397] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.130s user 0.008s sys 0.007s
I20250112 20:58:32.371810 18718 consensus_queue.cc:1035] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:32.373803 18664 raft_consensus.cc:357] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.374318 18664 raft_consensus.cc:383] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.374553 18664 raft_consensus.cc:738] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.375061 18664 consensus_queue.cc:260] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.372123 18774 raft_consensus.cc:513] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.376816 18435 leader_election.cc:304] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:32.377714 18718 raft_consensus.cc:2798] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:32.378037 18718 raft_consensus.cc:491] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:32.378340 18718 raft_consensus.cc:3054] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.378392 18774 leader_election.cc:290] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:32.379945 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219a31736df4499fb96cacfe05747d1a" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:32.385129 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219a31736df4499fb96cacfe05747d1a" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:32.386940 18362 leader_election.cc:343] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.390489 18718 raft_consensus.cc:513] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:32.396114 18360 leader_election.cc:343] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.399049 18360 leader_election.cc:304] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.401829 18664 ts_tablet_manager.cc:1428] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20250112 20:58:32.397061 18663 raft_consensus.cc:357] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.404276 18663 raft_consensus.cc:383] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.405012 18663 raft_consensus.cc:738] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.405925 18663 consensus_queue.cc:260] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.403991 18664 tablet_bootstrap.cc:492] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:32.403962 18865 raft_consensus.cc:2743] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.423472 18662 tablet_bootstrap.cc:654] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.432183 18663 ts_tablet_manager.cc:1428] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.060s user 0.007s sys 0.000s
I20250112 20:58:32.437274 18663 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:32.441679 18664 tablet_bootstrap.cc:654] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.452031 18663 tablet_bootstrap.cc:654] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.468552 18923 raft_consensus.cc:491] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.470335 18718 leader_election.cc:290] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:32.470172 18923 raft_consensus.cc:513] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.472146 18661 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:32.472805 18661 ts_tablet_manager.cc:1397] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.154s user 0.013s sys 0.016s
I20250112 20:58:32.479300 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:32.480134 18396 raft_consensus.cc:3054] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.474715 18940 raft_consensus.cc:491] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.490033 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:32.506079 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:32.507575 18923 leader_election.cc:290] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
W20250112 20:58:32.495864 18584 leader_election.cc:343] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.511569 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
W20250112 20:58:32.513257 18583 leader_election.cc:343] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.513689 18583 leader_election.cc:304] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:32.514457 18923 raft_consensus.cc:2743] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.494220 18940 raft_consensus.cc:513] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.521900 18662 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:32.522367 18662 ts_tablet_manager.cc:1397] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.206s user 0.009s sys 0.034s
I20250112 20:58:32.524739 18662 raft_consensus.cc:357] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.525271 18396 raft_consensus.cc:2463] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:32.525463 18662 raft_consensus.cc:383] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.525876 18662 raft_consensus.cc:738] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
W20250112 20:58:32.526294 18436 leader_election.cc:343] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.526520 18662 consensus_queue.cc:260] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.527843 18664 tablet_bootstrap.cc:492] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:32.527750 18435 leader_election.cc:304] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:32.518993 18661 raft_consensus.cc:357] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.528712 18661 raft_consensus.cc:383] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.528863 18662 ts_tablet_manager.cc:1428] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20250112 20:58:32.528353 18664 ts_tablet_manager.cc:1397] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.125s user 0.028s sys 0.026s
I20250112 20:58:32.529626 18893 raft_consensus.cc:2798] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:32.529764 18662 tablet_bootstrap.cc:492] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:32.530162 18893 raft_consensus.cc:695] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:32.531574 18664 raft_consensus.cc:357] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.533409 18664 raft_consensus.cc:383] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.533721 18664 raft_consensus.cc:738] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.534387 18664 consensus_queue.cc:260] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.535864 18662 tablet_bootstrap.cc:654] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.532989 18661 raft_consensus.cc:738] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.536680 18661 consensus_queue.cc:260] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.538656 18661 ts_tablet_manager.cc:1428] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.065s user 0.006s sys 0.000s
I20250112 20:58:32.540624 18661 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:32.541167 18664 ts_tablet_manager.cc:1428] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20250112 20:58:32.545450 18664 tablet_bootstrap.cc:492] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:32.558845 18940 leader_election.cc:290] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:32.561488 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:32.561985 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:32.564584 18664 tablet_bootstrap.cc:654] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.566921 18940 raft_consensus.cc:491] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.567608 18940 raft_consensus.cc:513] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.568737 18661 tablet_bootstrap.cc:654] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.570175 18940 leader_election.cc:290] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:32.571985 18509 leader_election.cc:304] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:32.574540 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af0c89ea5b0d4655959ccaad0b838bf4" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.575419 18618 raft_consensus.cc:2463] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:32.577491 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af0c89ea5b0d4655959ccaad0b838bf4" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:32.577778 18509 leader_election.cc:304] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:32.579062 18880 raft_consensus.cc:2743] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.579872 18880 raft_consensus.cc:2798] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:32.580271 18880 raft_consensus.cc:491] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:32.580693 18880 raft_consensus.cc:3054] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:32.585391 18509 leader_election.cc:343] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.531045 18893 consensus_queue.cc:237] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.603785 18880 raft_consensus.cc:513] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.606900 18618 raft_consensus.cc:3054] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.607374 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af0c89ea5b0d4655959ccaad0b838bf4" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:32.608192 18617 raft_consensus.cc:3054] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.624495 18617 raft_consensus.cc:2463] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:32.626088 18509 leader_election.cc:304] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:32.628305 18940 raft_consensus.cc:2798] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:32.629832 18940 raft_consensus.cc:695] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:32.633669 18940 consensus_queue.cc:237] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
W20250112 20:58:32.676451 18436 consensus_peers.cc:487] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:32.702216 18880 leader_election.cc:290] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:32.704905 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af0c89ea5b0d4655959ccaad0b838bf4" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:32.693636 18294 catalog_manager.cc:5526] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
W20250112 20:58:32.708976 18509 leader_election.cc:343] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.709895 18291 catalog_manager.cc:5526] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:32.712766 18662 tablet_bootstrap.cc:492] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:32.713284 18662 ts_tablet_manager.cc:1397] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.184s user 0.048s sys 0.108s
I20250112 20:58:32.715669 18662 raft_consensus.cc:357] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.719579 18662 raft_consensus.cc:383] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.719883 18662 raft_consensus.cc:738] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.720520 18662 consensus_queue.cc:260] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.723546 18718 raft_consensus.cc:491] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.724023 18718 raft_consensus.cc:513] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.725821 18718 leader_election.cc:290] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:32.726411 18868 raft_consensus.cc:491] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.726879 18868 raft_consensus.cc:513] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.729727 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:32.730041 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.730301 18470 raft_consensus.cc:2463] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:32.732213 18362 leader_election.cc:304] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:32.732998 18774 raft_consensus.cc:2798] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:32.747466 18774 raft_consensus.cc:491] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:32.748078 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.748725 18774 raft_consensus.cc:3054] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.744976 18868 leader_election.cc:290] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:32.734149 18541 raft_consensus.cc:3054] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.750563 18664 tablet_bootstrap.cc:492] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:32.750995 18664 ts_tablet_manager.cc:1397] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.206s user 0.013s sys 0.007s
I20250112 20:58:32.733130 18661 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:32.752089 18661 ts_tablet_manager.cc:1397] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.212s user 0.015s sys 0.005s
I20250112 20:58:32.736428 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:32.756624 18774 raft_consensus.cc:513] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.749549 18662 ts_tablet_manager.cc:1428] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
W20250112 20:58:32.749624 18360 leader_election.cc:343] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.757777 18661 raft_consensus.cc:357] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.758483 18661 raft_consensus.cc:383] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.758769 18395 raft_consensus.cc:2388] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2f9ce9a0415c4546b9a764ac06cefb38 in current term 1: Already voted for candidate a45f68dd3d9043809f3bb06287f5d2d3 in this term.
W20250112 20:58:32.735575 18435 leader_election.cc:343] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.759629 18663 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:32.760083 18663 ts_tablet_manager.cc:1397] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.323s user 0.016s sys 0.012s
I20250112 20:58:32.759150 18662 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:32.761943 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:32.763065 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
W20250112 20:58:32.763218 18360 leader_election.cc:343] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.763617 18470 raft_consensus.cc:3054] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.764923 18923 raft_consensus.cc:491] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.765426 18923 raft_consensus.cc:513] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.767376 18923 leader_election.cc:290] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:32.770970 18470 raft_consensus.cc:2463] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:32.771700 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:32.772662 18362 leader_election.cc:304] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.774087 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:32.774251 18719 raft_consensus.cc:2798] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:32.774762 18719 raft_consensus.cc:695] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
W20250112 20:58:32.775378 18584 leader_election.cc:343] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.775542 18719 consensus_queue.cc:237] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:32.776978 18435 leader_election.cc:304] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.777855 18824 raft_consensus.cc:2743] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:58:32.779011 18585 leader_election.cc:343] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.779515 18585 leader_election.cc:304] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
W20250112 20:58:32.771349 18342 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:58:32.780257 18923 raft_consensus.cc:2743] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.781440 18774 leader_election.cc:290] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:32.784004 18824 consensus_queue.cc:1035] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:32.758745 18661 raft_consensus.cc:738] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.783663 18662 tablet_bootstrap.cc:654] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.787456 18661 consensus_queue.cc:260] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.788401 18664 raft_consensus.cc:357] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.789117 18664 raft_consensus.cc:383] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.789407 18664 raft_consensus.cc:738] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.789989 18664 consensus_queue.cc:260] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.794584 18664 ts_tablet_manager.cc:1428] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20250112 20:58:32.795953 18664 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:32.807574 18662 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:32.808235 18662 ts_tablet_manager.cc:1397] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.049s user 0.010s sys 0.002s
I20250112 20:58:32.811218 18662 raft_consensus.cc:357] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.812142 18662 raft_consensus.cc:383] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.812458 18662 raft_consensus.cc:738] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.813139 18662 consensus_queue.cc:260] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:32.815723 18662 ts_tablet_manager.cc:1428] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250112 20:58:32.816710 18662 tablet_bootstrap.cc:492] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:32.818790 18664 tablet_bootstrap.cc:654] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.828719 18662 tablet_bootstrap.cc:654] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.834470 18664 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:32.835016 18664 ts_tablet_manager.cc:1397] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.002s
I20250112 20:58:32.838418 18662 tablet_bootstrap.cc:492] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:32.838979 18662 ts_tablet_manager.cc:1397] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.005s
I20250112 20:58:32.841562 18662 raft_consensus.cc:357] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.843209 18688 raft_consensus.cc:491] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.843808 18688 raft_consensus.cc:513] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.845170 18664 raft_consensus.cc:357] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.845851 18664 raft_consensus.cc:383] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.846216 18664 raft_consensus.cc:738] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.846746 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8300b885639841769a76417fb45f2db7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.849911 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8300b885639841769a76417fb45f2db7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:32.846796 18664 consensus_queue.cc:260] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.851678 18511 leader_election.cc:304] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.845777 18688 leader_election.cc:290] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:32.858342 18664 ts_tablet_manager.cc:1428] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20250112 20:58:32.858896 18880 raft_consensus.cc:2743] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.859277 18664 tablet_bootstrap.cc:492] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:32.842602 18662 raft_consensus.cc:383] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.861023 18662 raft_consensus.cc:738] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.861933 18662 consensus_queue.cc:260] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.864714 18662 ts_tablet_manager.cc:1428] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20250112 20:58:32.865701 18662 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:32.871627 18662 tablet_bootstrap.cc:654] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.876188 18663 raft_consensus.cc:357] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.876983 18663 raft_consensus.cc:383] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.877341 18663 raft_consensus.cc:738] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.878278 18663 consensus_queue.cc:260] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:32.879316 18664 tablet_bootstrap.cc:654] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.892877 18824 raft_consensus.cc:491] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.903628 18824 raft_consensus.cc:513] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.906834 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:32.906968 18824 leader_election.cc:290] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:32.907161 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:32.907651 18661 ts_tablet_manager.cc:1428] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.155s user 0.007s sys 0.000s
I20250112 20:58:32.907750 18614 raft_consensus.cc:2463] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
W20250112 20:58:32.908344 18436 leader_election.cc:343] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.908679 18661 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:32.907794 18664 tablet_bootstrap.cc:492] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:32.909301 18435 leader_election.cc:304] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:32.909821 18664 ts_tablet_manager.cc:1397] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.004s
I20250112 20:58:32.910604 18824 raft_consensus.cc:2798] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:32.911007 18824 raft_consensus.cc:491] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:32.911408 18824 raft_consensus.cc:3054] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.921429 18824 raft_consensus.cc:513] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:32.925141 18583 consensus_peers.cc:487] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250112 20:58:32.963850 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:32.959254 18718 raft_consensus.cc:491] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:32.982028 18614 raft_consensus.cc:3054] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:32.982288 18718 raft_consensus.cc:513] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.980935 18662 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:32.982957 18661 tablet_bootstrap.cc:654] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:32.983446 18662 ts_tablet_manager.cc:1397] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.118s user 0.019s sys 0.028s
I20250112 20:58:32.984683 18663 ts_tablet_manager.cc:1428] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.224s user 0.009s sys 0.002s
I20250112 20:58:32.985153 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:32.985795 18663 tablet_bootstrap.cc:492] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
W20250112 20:58:32.986621 18436 leader_election.cc:343] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.927512 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:32.953099 18290 catalog_manager.cc:5526] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:32.989537 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:32.989789 18436 leader_election.cc:343] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:32.990715 18435 leader_election.cc:343] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:32.991147 18435 leader_election.cc:304] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:32.991942 18870 raft_consensus.cc:2743] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:32.991865 18664 raft_consensus.cc:357] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:32.992836 18664 raft_consensus.cc:383] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:32.993167 18664 raft_consensus.cc:738] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.993408 18718 leader_election.cc:290] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:32.998524 18614 raft_consensus.cc:2463] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:32.999097 18664 consensus_queue.cc:260] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.001643 18664 ts_tablet_manager.cc:1428] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.091s user 0.004s sys 0.000s
I20250112 20:58:33.002463 18664 tablet_bootstrap.cc:492] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:32.999920 18662 raft_consensus.cc:357] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.004770 18662 raft_consensus.cc:383] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.005039 18662 raft_consensus.cc:738] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:32.969036 18824 leader_election.cc:290] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.008747 18664 tablet_bootstrap.cc:654] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.011926 18435 leader_election.cc:304] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:33.012542 18718 raft_consensus.cc:2798] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:33.013149 18718 raft_consensus.cc:695] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:33.013948 18718 consensus_queue.cc:237] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.022755 18541 raft_consensus.cc:3054] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.032069 18664 tablet_bootstrap.cc:492] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:33.032655 18664 ts_tablet_manager.cc:1397] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.004s
I20250112 20:58:33.035239 18688 raft_consensus.cc:491] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.035374 18664 raft_consensus.cc:357] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.035854 18688 raft_consensus.cc:513] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.036226 18664 raft_consensus.cc:383] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.036674 18664 raft_consensus.cc:738] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.037268 18664 consensus_queue.cc:260] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.037734 18688 leader_election.cc:290] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.039916 18664 ts_tablet_manager.cc:1428] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:58:33.039995 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d5dc78c20741968c7ca8f7ecec1c07" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:33.040789 18395 raft_consensus.cc:2463] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:33.041021 18664 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:33.042766 18509 leader_election.cc:304] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:33.043567 18880 raft_consensus.cc:2798] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:33.043933 18880 raft_consensus.cc:491] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:33.044272 18880 raft_consensus.cc:3054] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.047333 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d5dc78c20741968c7ca8f7ecec1c07" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:33.051712 18509 leader_election.cc:343] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.056671 18880 raft_consensus.cc:513] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.059458 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d5dc78c20741968c7ca8f7ecec1c07" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
W20250112 20:58:33.063625 18509 leader_election.cc:343] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.069234 18688 consensus_queue.cc:1035] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:33.072968 18663 tablet_bootstrap.cc:654] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.075531 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d5dc78c20741968c7ca8f7ecec1c07" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:33.076134 18395 raft_consensus.cc:3054] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.076388 18662 consensus_queue.cc:260] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.079125 18662 ts_tablet_manager.cc:1428] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.095s user 0.012s sys 0.000s
I20250112 20:58:33.080070 18662 tablet_bootstrap.cc:492] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:33.081112 18688 raft_consensus.cc:491] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.081670 18688 raft_consensus.cc:513] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.085747 18395 raft_consensus.cc:2463] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:33.086356 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:33.087074 18661 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:33.087160 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:33.089449 18661 ts_tablet_manager.cc:1397] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.181s user 0.012s sys 0.001s
W20250112 20:58:33.089951 18511 leader_election.cc:343] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.091531 18661 raft_consensus.cc:357] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.092012 18661 raft_consensus.cc:383] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.092208 18661 raft_consensus.cc:738] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.092722 18661 consensus_queue.cc:260] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.087510 18663 tablet_bootstrap.cc:492] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:33.084434 18293 catalog_manager.cc:5526] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:33.096374 18663 ts_tablet_manager.cc:1397] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.111s user 0.011s sys 0.000s
I20250112 20:58:33.088433 18541 raft_consensus.cc:3054] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.098485 18880 leader_election.cc:290] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.098619 18663 raft_consensus.cc:357] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.099282 18663 raft_consensus.cc:383] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.099524 18663 raft_consensus.cc:738] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.100186 18663 consensus_queue.cc:260] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.101958 18664 tablet_bootstrap.cc:654] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.102164 18509 leader_election.cc:304] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:33.102677 18663 ts_tablet_manager.cc:1428] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20250112 20:58:33.103281 18509 leader_election.cc:343] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.103622 18663 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:33.103716 18509 leader_election.cc:304] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:33.105782 18940 raft_consensus.cc:2798] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:33.106240 18940 raft_consensus.cc:695] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:33.106971 18940 consensus_queue.cc:237] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.110775 18663 tablet_bootstrap.cc:654] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.110759 18938 raft_consensus.cc:2743] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:33.088562 18688 leader_election.cc:290] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:33.120077 18718 consensus_queue.cc:1035] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:33.121752 18662 tablet_bootstrap.cc:654] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.125187 18661 ts_tablet_manager.cc:1428] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.035s user 0.005s sys 0.001s
I20250112 20:58:33.126062 18661 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:33.132395 18661 tablet_bootstrap.cc:654] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.142020 18662 tablet_bootstrap.cc:492] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:33.142530 18662 ts_tablet_manager.cc:1397] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.000s
I20250112 20:58:33.142874 18688 raft_consensus.cc:491] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.143435 18688 raft_consensus.cc:513] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:33.146186 18509 consensus_peers.cc:487] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:33.150707 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:33.152923 18469 raft_consensus.cc:2463] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:33.154145 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:33.157440 18509 leader_election.cc:343] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.150625 18662 raft_consensus.cc:357] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.151898 18688 leader_election.cc:290] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
W20250112 20:58:33.152016 18436 consensus_peers.cc:487] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:33.161015 18511 leader_election.cc:304] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:33.162467 18880 raft_consensus.cc:2798] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:33.162951 18880 raft_consensus.cc:491] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:33.163407 18880 raft_consensus.cc:3054] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.172116 18664 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:33.172549 18664 ts_tablet_manager.cc:1397] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.132s user 0.013s sys 0.004s
I20250112 20:58:33.159636 18662 raft_consensus.cc:383] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.176384 18880 raft_consensus.cc:513] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.179060 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:33.179603 18469 raft_consensus.cc:3054] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.180292 18664 raft_consensus.cc:357] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.181051 18664 raft_consensus.cc:383] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.181349 18664 raft_consensus.cc:738] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.181986 18664 consensus_queue.cc:260] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.186681 18469 raft_consensus.cc:2463] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:33.189841 18774 raft_consensus.cc:491] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.191897 18774 raft_consensus.cc:513] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.194025 18774 leader_election.cc:290] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.174922 18662 raft_consensus.cc:738] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.194974 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8820dfc60bc477985f861406c22dd3b" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.196081 18664 ts_tablet_manager.cc:1428] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20250112 20:58:33.195787 18662 consensus_queue.cc:260] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:33.196408 18360 leader_election.cc:343] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.197893 18662 ts_tablet_manager.cc:1428] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.055s user 0.007s sys 0.000s
I20250112 20:58:33.199285 18511 leader_election.cc:304] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:33.199077 18290 catalog_manager.cc:5526] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:33.199986 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:33.196219 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8820dfc60bc477985f861406c22dd3b" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:33.201247 18662 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
W20250112 20:58:33.202211 18361 leader_election.cc:343] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:33.202267 18509 leader_election.cc:343] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.202658 18361 leader_election.cc:304] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
W20250112 20:58:33.203339 18360 consensus_peers.cc:487] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 -> Peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Couldn't send request to peer c60c0cbe377c450297e3d536b7f7c89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:33.203593 18880 leader_election.cc:290] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.203990 18719 raft_consensus.cc:2743] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:33.204384 18880 raft_consensus.cc:2798] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:33.204888 18880 raft_consensus.cc:695] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:33.205686 18664 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:33.205639 18880 consensus_queue.cc:237] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.210804 18661 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:33.210979 18923 raft_consensus.cc:491] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.211301 18661 ts_tablet_manager.cc:1397] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.085s user 0.011s sys 0.006s
I20250112 20:58:33.211546 18923 raft_consensus.cc:513] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.213526 18923 leader_election.cc:290] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.250522 18718 raft_consensus.cc:491] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.255802 18718 raft_consensus.cc:513] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.215556 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:33.216511 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
W20250112 20:58:33.258026 18583 leader_election.cc:343] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.258337 18661 raft_consensus.cc:357] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.258633 18718 leader_election.cc:290] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
W20250112 20:58:33.259708 18584 leader_election.cc:343] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.260188 18774 raft_consensus.cc:491] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.260187 18584 leader_election.cc:304] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:33.260557 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985f9b21969e48739677e3cdc86e25de" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:33.261485 18661 raft_consensus.cc:383] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.256515 18662 tablet_bootstrap.cc:654] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.261809 18923 raft_consensus.cc:2743] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:58:33.262560 18435 leader_election.cc:343] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.263320 18774 raft_consensus.cc:513] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.263661 18663 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:33.264184 18663 ts_tablet_manager.cc:1397] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.161s user 0.013s sys 0.009s
I20250112 20:58:33.261802 18661 raft_consensus.cc:738] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.265067 18661 consensus_queue.cc:260] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.265892 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:33.267115 18688 raft_consensus.cc:491] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.267056 18663 raft_consensus.cc:357] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.267640 18688 raft_consensus.cc:513] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.267906 18663 raft_consensus.cc:383] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.268266 18663 raft_consensus.cc:738] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.268976 18663 consensus_queue.cc:260] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.269547 18661 ts_tablet_manager.cc:1428] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.058s user 0.004s sys 0.004s
W20250112 20:58:33.270032 18362 leader_election.cc:343] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.270964 18688 leader_election.cc:290] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.271814 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe90092ab277453fb41e05a8be798304" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:33.273144 18469 raft_consensus.cc:2463] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:33.271850 18662 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:33.274125 18774 leader_election.cc:290] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:33.274257 18662 ts_tablet_manager.cc:1397] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.073s user 0.010s sys 0.000s
I20250112 20:58:33.274371 18511 leader_election.cc:304] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:33.276264 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe90092ab277453fb41e05a8be798304" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.276885 18688 raft_consensus.cc:2798] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:33.277230 18688 raft_consensus.cc:491] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:33.277498 18688 raft_consensus.cc:3054] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.278537 18661 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:33.283865 18661 tablet_bootstrap.cc:654] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.286023 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985f9b21969e48739677e3cdc86e25de" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:33.288580 18663 ts_tablet_manager.cc:1428] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.024s user 0.002s sys 0.003s
I20250112 20:58:33.284245 18662 raft_consensus.cc:357] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.289806 18663 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:33.289916 18662 raft_consensus.cc:383] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.290324 18662 raft_consensus.cc:738] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.291694 18662 consensus_queue.cc:260] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.293066 18718 raft_consensus.cc:491] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250112 20:58:33.294684 18509 leader_election.cc:343] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.295186 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:33.296552 18543 raft_consensus.cc:2463] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
W20250112 20:58:33.297814 18436 leader_election.cc:343] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.298205 18436 leader_election.cc:304] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:33.299043 18824 raft_consensus.cc:2743] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:33.299023 18361 leader_election.cc:304] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38
I20250112 20:58:33.299875 18774 raft_consensus.cc:2798] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:33.300210 18774 raft_consensus.cc:491] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:33.300278 18688 raft_consensus.cc:513] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.300587 18774 raft_consensus.cc:3054] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.301507 18664 tablet_bootstrap.cc:654] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.293474 18718 raft_consensus.cc:513] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.304597 18662 ts_tablet_manager.cc:1428] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20250112 20:58:33.305744 18662 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:33.290505 18293 catalog_manager.cc:5526] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:33.322607 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219a31736df4499fb96cacfe05747d1a" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.324244 18688 leader_election.cc:290] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.334110 18661 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:33.334565 18661 ts_tablet_manager.cc:1397] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.056s user 0.004s sys 0.009s
I20250112 20:58:33.331104 18718 leader_election.cc:290] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
W20250112 20:58:33.329802 18435 leader_election.cc:343] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.338456 18719 raft_consensus.cc:491] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.341032 18719 raft_consensus.cc:513] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.341576 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219a31736df4499fb96cacfe05747d1a" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:33.342296 18395 raft_consensus.cc:2463] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:33.342216 18774 raft_consensus.cc:513] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.343116 18719 leader_election.cc:290] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.343804 18435 leader_election.cc:304] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:33.344281 18774 leader_election.cc:290] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
W20250112 20:58:33.330251 18584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.131:39861, user_credentials={real_user=slave}} blocked reactor thread for 68808.3us
I20250112 20:58:33.340073 18663 tablet_bootstrap.cc:654] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.344586 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "684a3c0a345f4ce483768dfe1572ca13" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.353435 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe90092ab277453fb41e05a8be798304" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:33.354182 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:33.354751 18540 raft_consensus.cc:3054] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.339812 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe90092ab277453fb41e05a8be798304" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:33.354197 18469 raft_consensus.cc:3054] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.350476 18870 raft_consensus.cc:2798] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20250112 20:58:33.370792 18509 leader_election.cc:343] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.370929 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:33.371544 18664 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:33.371948 18664 ts_tablet_manager.cc:1397] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.166s user 0.018s sys 0.017s
I20250112 20:58:33.376778 18662 tablet_bootstrap.cc:654] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.378729 18540 raft_consensus.cc:2463] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:33.363428 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "684a3c0a345f4ce483768dfe1572ca13" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:33.365064 18870 raft_consensus.cc:491] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:33.383203 18870 raft_consensus.cc:3054] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.383283 18664 raft_consensus.cc:357] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.384037 18664 raft_consensus.cc:383] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.384276 18664 raft_consensus.cc:738] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.384847 18664 consensus_queue.cc:260] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.383577 18361 leader_election.cc:304] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:33.387284 18774 raft_consensus.cc:2743] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250112 20:58:33.382179 18362 leader_election.cc:343] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.390132 18469 raft_consensus.cc:2463] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:33.391290 18511 leader_election.cc:304] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:33.393151 18880 raft_consensus.cc:2798] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:33.393651 18880 raft_consensus.cc:695] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:33.394572 18880 consensus_queue.cc:237] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.396023 18664 ts_tablet_manager.cc:1428] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.024s user 0.007s sys 0.000s
I20250112 20:58:33.390178 18661 raft_consensus.cc:357] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.398566 18661 raft_consensus.cc:383] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.398820 18661 raft_consensus.cc:738] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.399165 18774 consensus_queue.cc:1035] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:33.399384 18661 consensus_queue.cc:260] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.401504 18661 ts_tablet_manager.cc:1428] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.067s user 0.006s sys 0.000s
I20250112 20:58:33.402889 18664 tablet_bootstrap.cc:492] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:33.404834 18661 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:33.410084 18361 leader_election.cc:304] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38
I20250112 20:58:33.413406 18866 raft_consensus.cc:2798] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:33.413836 18866 raft_consensus.cc:695] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:33.416249 18663 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:33.416815 18663 ts_tablet_manager.cc:1397] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.127s user 0.031s sys 0.008s
I20250112 20:58:33.417483 18664 tablet_bootstrap.cc:654] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.416844 18866 consensus_queue.cc:237] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.419809 18663 raft_consensus.cc:357] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.420681 18663 raft_consensus.cc:383] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.421008 18663 raft_consensus.cc:738] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.421849 18663 consensus_queue.cc:260] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.425398 18870 raft_consensus.cc:513] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.430815 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219a31736df4499fb96cacfe05747d1a" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:33.431546 18617 raft_consensus.cc:3054] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.433095 18663 ts_tablet_manager.cc:1428] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20250112 20:58:33.434171 18663 tablet_bootstrap.cc:492] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:33.438882 18870 leader_election.cc:290] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.414098 18661 tablet_bootstrap.cc:654] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.441638 18617 raft_consensus.cc:2463] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:33.442970 18435 leader_election.cc:304] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:33.444160 18870 raft_consensus.cc:2798] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:33.444447 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "219a31736df4499fb96cacfe05747d1a" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:33.446581 18395 raft_consensus.cc:3054] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.448006 18663 tablet_bootstrap.cc:654] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.449101 18664 tablet_bootstrap.cc:492] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:33.449574 18664 ts_tablet_manager.cc:1397] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.047s user 0.007s sys 0.002s
I20250112 20:58:33.451952 18661 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:33.452411 18661 ts_tablet_manager.cc:1397] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.001s
I20250112 20:58:33.452451 18664 raft_consensus.cc:357] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.453317 18664 raft_consensus.cc:383] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.453570 18664 raft_consensus.cc:738] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.454878 18661 raft_consensus.cc:357] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.455567 18661 raft_consensus.cc:383] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.455823 18661 raft_consensus.cc:738] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.454324 18664 consensus_queue.cc:260] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.456403 18661 consensus_queue.cc:260] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.458344 18664 ts_tablet_manager.cc:1428] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250112 20:58:33.458729 18661 ts_tablet_manager.cc:1428] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:58:33.459226 18664 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:33.459589 18661 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:33.446583 18870 raft_consensus.cc:695] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:33.464233 18870 consensus_queue.cc:237] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.465669 18664 tablet_bootstrap.cc:654] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.466969 18663 tablet_bootstrap.cc:492] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:33.467412 18663 ts_tablet_manager.cc:1397] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.000s
I20250112 20:58:33.469837 18663 raft_consensus.cc:357] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.470484 18663 raft_consensus.cc:383] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.470741 18663 raft_consensus.cc:738] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.474366 18824 consensus_queue.cc:1035] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:33.477355 18663 consensus_queue.cc:260] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.478969 18661 tablet_bootstrap.cc:654] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.479463 18663 ts_tablet_manager.cc:1428] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20250112 20:58:33.481797 18663 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:33.484719 18664 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:33.485162 18664 ts_tablet_manager.cc:1397] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.026s user 0.014s sys 0.003s
I20250112 20:58:33.486828 18291 catalog_manager.cc:5526] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:33.488344 18663 tablet_bootstrap.cc:654] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.488209 18664 raft_consensus.cc:357] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.488950 18664 raft_consensus.cc:383] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.489358 18664 raft_consensus.cc:738] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.490414 18664 consensus_queue.cc:260] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.493738 18664 ts_tablet_manager.cc:1428] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250112 20:58:33.496726 18664 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:33.498692 18395 raft_consensus.cc:2463] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:33.501003 18719 raft_consensus.cc:491] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.501720 18719 raft_consensus.cc:513] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.510679 18664 tablet_bootstrap.cc:654] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.515547 18662 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:33.516122 18662 ts_tablet_manager.cc:1397] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.211s user 0.018s sys 0.016s
I20250112 20:58:33.520962 18662 raft_consensus.cc:357] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.521613 18662 raft_consensus.cc:383] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.521871 18662 raft_consensus.cc:738] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.522429 18662 consensus_queue.cc:260] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.525501 18664 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:33.526201 18664 ts_tablet_manager.cc:1397] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.030s user 0.011s sys 0.006s
I20250112 20:58:33.523918 18291 catalog_manager.cc:5526] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:33.528605 18664 raft_consensus.cc:357] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.529249 18664 raft_consensus.cc:383] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20250112 20:58:33.510499 18361 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.131:39861, user_credentials={real_user=slave}} blocked reactor thread for 86426.1us
I20250112 20:58:33.529495 18664 raft_consensus.cc:738] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.540773 18662 ts_tablet_manager.cc:1428] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.024s user 0.015s sys 0.001s
I20250112 20:58:33.531072 18294 catalog_manager.cc:5526] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:33.548411 18662 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:33.539887 18719 leader_election.cc:290] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.537458 18395 raft_consensus.cc:3054] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.543365 18923 raft_consensus.cc:491] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.542894 18824 raft_consensus.cc:491] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.552204 18664 consensus_queue.cc:260] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.555438 18664 ts_tablet_manager.cc:1428] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20250112 20:58:33.556572 18664 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:33.557375 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af0c89ea5b0d4655959ccaad0b838bf4" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:33.560078 18436 consensus_peers.cc:487] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:33.561565 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af0c89ea5b0d4655959ccaad0b838bf4" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.565481 18540 raft_consensus.cc:3054] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.572700 18824 raft_consensus.cc:513] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.574661 18824 leader_election.cc:290] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.576689 18923 raft_consensus.cc:513] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.574810 18661 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:33.578675 18923 leader_election.cc:290] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:33.578675 18361 leader_election.cc:304] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:33.579804 18661 ts_tablet_manager.cc:1397] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.120s user 0.022s sys 0.013s
I20250112 20:58:33.579674 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:33.580730 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:33.582427 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:33.532419 18362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 106743us
I20250112 20:58:33.583664 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:33.590442 18774 raft_consensus.cc:2743] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:33.591487 18394 raft_consensus.cc:2463] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
W20250112 20:58:33.592651 18435 leader_election.cc:343] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:33.593609 18585 leader_election.cc:343] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.595202 18435 leader_election.cc:304] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:33.596172 19017 raft_consensus.cc:2798] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:33.596545 19017 raft_consensus.cc:491] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:33.604001 18661 raft_consensus.cc:357] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.604771 18661 raft_consensus.cc:383] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.605011 18661 raft_consensus.cc:738] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.605568 18661 consensus_queue.cc:260] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.608582 18394 raft_consensus.cc:3054] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.611910 18664 tablet_bootstrap.cc:654] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.612924 18688 raft_consensus.cc:491] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250112 20:58:33.606390 18584 leader_election.cc:343] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:33.613941 18584 leader_election.cc:304] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:33.613524 18688 raft_consensus.cc:513] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.614976 18923 raft_consensus.cc:2743] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:33.616053 18688 leader_election.cc:290] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.596868 19017 raft_consensus.cc:3054] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.617494 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f52ac1efd0e417393e5127713a43a18" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.618129 18614 raft_consensus.cc:2463] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:33.620113 18509 leader_election.cc:304] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:33.621766 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f52ac1efd0e417393e5127713a43a18" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:33.622421 18392 raft_consensus.cc:2463] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:33.624003 18880 raft_consensus.cc:2798] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:33.624507 18880 raft_consensus.cc:491] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:33.634461 18880 raft_consensus.cc:3054] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.636615 18663 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:33.637188 18663 ts_tablet_manager.cc:1397] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.156s user 0.030s sys 0.028s
I20250112 20:58:33.632323 18661 ts_tablet_manager.cc:1428] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.045s user 0.004s sys 0.000s
I20250112 20:58:33.634130 19017 raft_consensus.cc:513] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.639071 18661 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:33.637317 18986 consensus_queue.cc:1035] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20250112 20:58:33.643426 18923 raft_consensus.cc:491] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.646816 18923 raft_consensus.cc:513] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.651203 18923 leader_election.cc:290] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.651881 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d5dc78c20741968c7ca8f7ecec1c07" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:33.655788 18663 raft_consensus.cc:357] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.656520 18663 raft_consensus.cc:383] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.656806 18663 raft_consensus.cc:738] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.657490 19016 raft_consensus.cc:491] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.657407 18663 consensus_queue.cc:260] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:33.659983 18663 ts_tablet_manager.cc:1428] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20250112 20:58:33.660964 18662 tablet_bootstrap.cc:654] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.660965 18663 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:33.657931 19016 raft_consensus.cc:513] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.667915 19016 leader_election.cc:290] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.668520 19017 leader_election.cc:290] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.670977 18664 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:33.671420 18664 ts_tablet_manager.cc:1397] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.115s user 0.011s sys 0.002s
I20250112 20:58:33.672359 18661 tablet_bootstrap.cc:654] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.673856 18663 tablet_bootstrap.cc:654] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.673872 18664 raft_consensus.cc:357] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.676779 18664 raft_consensus.cc:383] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.677109 18880 raft_consensus.cc:513] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.677574 18664 raft_consensus.cc:738] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.678457 18664 consensus_queue.cc:260] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.679524 18880 leader_election.cc:290] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.680167 18664 ts_tablet_manager.cc:1428] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250112 20:58:33.767242 18923 raft_consensus.cc:491] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.774724 18393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f52ac1efd0e417393e5127713a43a18" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:33.777954 18870 raft_consensus.cc:491] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.791512 18661 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:33.735821 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:33.804719 18540 raft_consensus.cc:2463] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:33.803241 18662 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:33.802950 18870 raft_consensus.cc:513] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.801266 18393 raft_consensus.cc:3054] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.823258 18688 raft_consensus.cc:491] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.824642 18688 raft_consensus.cc:513] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.828186 18688 leader_election.cc:290] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.832844 18923 raft_consensus.cc:513] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.834050 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f52ac1efd0e417393e5127713a43a18" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:33.834905 18618 raft_consensus.cc:3054] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.835275 18617 raft_consensus.cc:3054] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.836952 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:33.837874 18470 raft_consensus.cc:2463] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:33.850688 18664 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:33.852974 18661 ts_tablet_manager.cc:1397] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.215s user 0.014s sys 0.000s
I20250112 20:58:33.839558 18585 leader_election.cc:304] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:33.857666 18986 raft_consensus.cc:491] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.858304 18986 raft_consensus.cc:513] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:33.861301 18662 ts_tablet_manager.cc:1397] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.313s user 0.015s sys 0.001s
I20250112 20:58:33.867854 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d5dc78c20741968c7ca8f7ecec1c07" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:33.873416 18995 raft_consensus.cc:2798] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:33.873847 18995 raft_consensus.cc:491] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:33.874179 18995 raft_consensus.cc:3054] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.875322 18664 tablet_bootstrap.cc:654] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.880230 18940 raft_consensus.cc:491] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.881244 18940 raft_consensus.cc:513] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.884058 18940 leader_election.cc:290] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:33.892083 18662 raft_consensus.cc:357] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.893074 18662 raft_consensus.cc:383] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.893399 18662 raft_consensus.cc:738] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:33.860443 18986 leader_election.cc:290] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:33.896813 19037 raft_consensus.cc:491] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:33.896986 18393 raft_consensus.cc:2463] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:33.897511 19037 raft_consensus.cc:513] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:33.890776 19024 log.cc:926] Time spent T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20250112 20:58:33.899787 19037 leader_election.cc:290] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:33.894109 18662 consensus_queue.cc:260] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.914901 18870 leader_election.cc:290] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:33.917012 18661 raft_consensus.cc:357] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.918118 18661 raft_consensus.cc:383] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:33.918529 18661 raft_consensus.cc:738] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
W20250112 20:58:33.919724 18362 consensus_peers.cc:487] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:33.919394 18661 consensus_queue.cc:260] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:33.922127 18661 ts_tablet_manager.cc:1428] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.069s user 0.008s sys 0.000s
I20250112 20:58:33.923527 18661 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:33.932639 18663 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:33.933343 18661 tablet_bootstrap.cc:654] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:33.933951 18663 ts_tablet_manager.cc:1397] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.273s user 0.024s sys 0.010s
I20250112 20:58:33.946077 18662 ts_tablet_manager.cc:1428] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.084s user 0.009s sys 0.000s
I20250112 20:58:33.947456 18662 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:33.948225 18509 leader_election.cc:304] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:33.951539 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8820dfc60bc477985f861406c22dd3b" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.953760 18615 raft_consensus.cc:3054] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.954849 18995 raft_consensus.cc:513] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:33.943547 18585 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 104218us
I20250112 20:58:33.962203 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:33.963034 18469 raft_consensus.cc:3054] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:33.964380 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.965123 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:33.966159 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.965224 18614 raft_consensus.cc:2463] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:33.967094 18616 raft_consensus.cc:2388] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 1: Already voted for candidate c60c0cbe377c450297e3d536b7f7c89c in this term.
I20250112 20:58:33.966684 18611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:33.986083 18611 raft_consensus.cc:2388] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2f9ce9a0415c4546b9a764ac06cefb38 in current term 1: Already voted for candidate c60c0cbe377c450297e3d536b7f7c89c in this term.
I20250112 20:58:33.978467 18940 raft_consensus.cc:2798] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:33.980154 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:33.988308 18940 raft_consensus.cc:695] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:33.984934 18436 leader_election.cc:304] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:33.989578 18940 consensus_queue.cc:237] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:33.997593 18469 raft_consensus.cc:2463] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:33.998193 18870 raft_consensus.cc:2758] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20250112 20:58:33.998325 18664 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:33.998996 18664 ts_tablet_manager.cc:1397] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.149s user 0.011s sys 0.021s
I20250112 20:58:33.995532 18923 leader_election.cc:290] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.003504 18540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:34.004247 18540 raft_consensus.cc:2463] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
W20250112 20:58:34.021394 18509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 49110.5us
I20250112 20:58:34.017306 18995 leader_election.cc:290] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.018487 19036 raft_consensus.cc:491] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.029302 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:34.030146 18545 raft_consensus.cc:3054] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.029804 19036 raft_consensus.cc:513] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:34.006434 18585 leader_election.cc:343] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:34.031690 18509 leader_election.cc:343] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.034215 18469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2806263c3471cac0496b194a73ab9" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.038331 18469 raft_consensus.cc:2388] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 1: Already voted for candidate c60c0cbe377c450297e3d536b7f7c89c in this term.
W20250112 20:58:34.036244 18435 leader_election.cc:343] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.040457 18509 leader_election.cc:304] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:34.041396 18662 tablet_bootstrap.cc:654] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.045795 18585 leader_election.cc:304] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:34.050114 18545 raft_consensus.cc:2463] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:34.044503 18618 raft_consensus.cc:2463] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:34.062346 18923 raft_consensus.cc:2798] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:34.056872 18663 raft_consensus.cc:357] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.072096 18663 raft_consensus.cc:383] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.062412 18688 raft_consensus.cc:2798] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:34.072579 18663 raft_consensus.cc:738] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.072901 18688 raft_consensus.cc:491] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:34.073312 18688 raft_consensus.cc:3054] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.073559 18880 consensus_queue.cc:1035] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20250112 20:58:34.073482 18663 consensus_queue.cc:260] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.074888 18661 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:34.075345 18661 ts_tablet_manager.cc:1397] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.152s user 0.032s sys 0.015s
I20250112 20:58:34.075187 18880 consensus_queue.cc:1035] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:34.076267 18663 ts_tablet_manager.cc:1428] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.132s user 0.007s sys 0.000s
I20250112 20:58:34.071208 18664 raft_consensus.cc:357] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.078665 18664 raft_consensus.cc:383] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.079023 18664 raft_consensus.cc:738] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.076773 18923 raft_consensus.cc:695] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:34.092815 18663 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:34.094491 18664 consensus_queue.cc:260] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.090307 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdbd6c5512664ac7a36a13bd1a89d75f" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:34.100746 18923 consensus_queue.cc:237] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:34.099174 18544 raft_consensus.cc:2463] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:34.104355 18664 ts_tablet_manager.cc:1428] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.105s user 0.007s sys 0.002s
I20250112 20:58:34.106009 18664 tablet_bootstrap.cc:492] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:34.107745 18661 raft_consensus.cc:357] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.108785 18661 raft_consensus.cc:383] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.109071 18661 raft_consensus.cc:738] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.109719 18661 consensus_queue.cc:260] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.111799 18661 ts_tablet_manager.cc:1428] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.036s user 0.007s sys 0.000s
I20250112 20:58:34.112877 18661 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:34.122964 18661 tablet_bootstrap.cc:654] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.130088 18940 consensus_queue.cc:1035] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250112 20:58:34.133773 18511 leader_election.cc:304] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
W20250112 20:58:34.135118 18509 consensus_peers.cc:487] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb -> Peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Couldn't send request to peer c60c0cbe377c450297e3d536b7f7c89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:34.140214 18940 raft_consensus.cc:2743] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.142100 18662 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:34.142621 18662 ts_tablet_manager.cc:1397] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.195s user 0.014s sys 0.017s
I20250112 20:58:34.155998 18662 raft_consensus.cc:357] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.152707 18940 consensus_queue.cc:1035] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20250112 20:58:34.156562 18662 raft_consensus.cc:383] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.158244 18662 raft_consensus.cc:738] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.161005 18436 leader_election.cc:304] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:34.161981 19017 raft_consensus.cc:2798] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:34.162338 19017 raft_consensus.cc:491] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:34.162698 19017 raft_consensus.cc:3054] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:34.164572 18360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 76045.7us
I20250112 20:58:34.168730 18662 consensus_queue.cc:260] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.177572 18664 tablet_bootstrap.cc:654] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.169718 18986 raft_consensus.cc:491] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.183704 18663 tablet_bootstrap.cc:654] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.183097 18986 raft_consensus.cc:513] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.186407 18986 leader_election.cc:290] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:34.189859 18583 leader_election.cc:304] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:34.192025 19037 raft_consensus.cc:2743] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.197690 19031 raft_consensus.cc:491] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.198208 19031 raft_consensus.cc:513] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.200388 19031 leader_election.cc:290] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:34.201972 18396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:34.202191 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6961f63ec790480aaa86f70431ed86c6" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.202752 18396 raft_consensus.cc:3054] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:34.203780 18362 leader_election.cc:343] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.200253 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.158833 18688 raft_consensus.cc:513] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.170902 19017 raft_consensus.cc:513] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.238087 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985f9b21969e48739677e3cdc86e25de" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.238937 18468 raft_consensus.cc:2463] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:34.242362 19036 leader_election.cc:290] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.243243 19017 leader_election.cc:290] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.246127 18688 leader_election.cc:290] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:34.248199 18662 ts_tablet_manager.cc:1428] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.105s user 0.009s sys 0.000s
I20250112 20:58:34.249316 18662 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:34.249078 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:34.232268 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdbd6c5512664ac7a36a13bd1a89d75f" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.251386 18893 consensus_queue.cc:1035] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:34.281085 18663 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:34.281858 18663 ts_tablet_manager.cc:1397] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.194s user 0.014s sys 0.014s
I20250112 20:58:34.284914 18661 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:34.307243 18661 ts_tablet_manager.cc:1397] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.195s user 0.016s sys 0.001s
I20250112 20:58:34.298535 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdbd6c5512664ac7a36a13bd1a89d75f" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:34.302517 18511 leader_election.cc:304] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:34.309451 18940 raft_consensus.cc:2798] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:34.309808 18940 raft_consensus.cc:491] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:34.310117 18940 raft_consensus.cc:3054] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.310832 18611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6961f63ec790480aaa86f70431ed86c6" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:34.311546 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdbd6c5512664ac7a36a13bd1a89d75f" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:34.312161 18545 raft_consensus.cc:3054] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.305423 18663 raft_consensus.cc:357] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.292563 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:34.318688 18940 raft_consensus.cc:513] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.320614 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.324077 18663 raft_consensus.cc:383] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.325489 18663 raft_consensus.cc:738] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.328152 18940 leader_election.cc:290] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
W20250112 20:58:34.328962 18583 leader_election.cc:343] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.328845 18663 consensus_queue.cc:260] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.329890 18391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8820dfc60bc477985f861406c22dd3b" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.329614 18583 leader_election.cc:304] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:34.330543 18391 raft_consensus.cc:2463] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:34.331841 18663 ts_tablet_manager.cc:1428] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.050s user 0.007s sys 0.002s
I20250112 20:58:34.332696 18663 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:34.333225 19037 raft_consensus.cc:2743] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.328848 18613 raft_consensus.cc:3054] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.350858 18396 raft_consensus.cc:2463] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:34.357597 18661 raft_consensus.cc:357] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.360098 18661 raft_consensus.cc:383] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.366909 18661 raft_consensus.cc:738] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.372728 18662 tablet_bootstrap.cc:654] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.377727 18661 consensus_queue.cc:260] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.384023 18661 ts_tablet_manager.cc:1428] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.076s user 0.006s sys 0.000s
I20250112 20:58:34.385023 18661 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:34.361129 18663 tablet_bootstrap.cc:654] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.362031 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985f9b21969e48739677e3cdc86e25de" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.395013 18390 raft_consensus.cc:2463] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:34.362324 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:34.395927 18395 raft_consensus.cc:3054] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.399451 18664 tablet_bootstrap.cc:492] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:34.399946 18664 ts_tablet_manager.cc:1397] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.294s user 0.017s sys 0.024s
W20250112 20:58:34.359094 18435 leader_election.cc:343] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.407127 18663 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
W20250112 20:58:34.407388 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 48330.4us
I20250112 20:58:34.407583 18663 ts_tablet_manager.cc:1397] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.075s user 0.012s sys 0.001s
I20250112 20:58:34.406183 18395 raft_consensus.cc:2463] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:34.410832 19017 consensus_queue.cc:1035] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20250112 20:58:34.414754 18664 raft_consensus.cc:357] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:34.538484 18664 raft_consensus.cc:383] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.538790 18664 raft_consensus.cc:738] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.539407 18664 consensus_queue.cc:260] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:34.363808 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985f9b21969e48739677e3cdc86e25de" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:34.540894 18394 raft_consensus.cc:3054] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.421120 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "985f9b21969e48739677e3cdc86e25de" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
W20250112 20:58:34.426317 18435 leader_election.cc:343] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.400099 18661 tablet_bootstrap.cc:654] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.546689 18664 ts_tablet_manager.cc:1428] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.134s user 0.006s sys 0.002s
I20250112 20:58:34.547323 18394 raft_consensus.cc:2463] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:34.547783 18664 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
W20250112 20:58:34.431839 18360 leader_election.cc:343] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.549588 18360 leader_election.cc:304] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
W20250112 20:58:34.550434 18360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 118603us
I20250112 20:58:34.551316 19059 raft_consensus.cc:2743] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.284283 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.477496 19060 raft_consensus.cc:491] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.508217 18392 raft_consensus.cc:2371] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1. Re-sending same reply.
I20250112 20:58:34.509125 19034 raft_consensus.cc:491] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.552981 19060 raft_consensus.cc:513] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.553417 19034 raft_consensus.cc:513] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.554939 19060 leader_election.cc:290] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.518318 18545 raft_consensus.cc:2463] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:34.555678 18467 raft_consensus.cc:3054] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.519766 18662 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:34.527987 18613 raft_consensus.cc:2463] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:34.556809 18664 tablet_bootstrap.cc:654] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.556730 19037 raft_consensus.cc:491] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.555197 19034 leader_election.cc:290] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.557333 19037 raft_consensus.cc:513] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.559295 19037 leader_election.cc:290] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
W20250112 20:58:34.544953 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 118661us
I20250112 20:58:34.560068 18435 leader_election.cc:304] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:34.560803 19034 raft_consensus.cc:2798] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:34.561270 19034 raft_consensus.cc:695] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:34.562088 19034 consensus_queue.cc:237] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.544911 18471 raft_consensus.cc:3054] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.569541 18471 raft_consensus.cc:2463] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:34.562397 18664 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:34.570443 18664 ts_tablet_manager.cc:1397] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.005s
I20250112 20:58:34.574410 18664 raft_consensus.cc:357] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.575078 18664 raft_consensus.cc:383] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.575331 18664 raft_consensus.cc:738] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.575897 18664 consensus_queue.cc:260] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.578876 18664 ts_tablet_manager.cc:1428] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20250112 20:58:34.579794 18664 tablet_bootstrap.cc:492] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:34.582755 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.584321 19007 raft_consensus.cc:491] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.584810 19007 raft_consensus.cc:513] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:34.586242 18583 leader_election.cc:343] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.586735 19007 leader_election.cc:290] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.556766 18662 ts_tablet_manager.cc:1397] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.308s user 0.015s sys 0.004s
I20250112 20:58:34.590317 18664 tablet_bootstrap.cc:654] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.591331 18662 raft_consensus.cc:357] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.545369 18995 consensus_queue.cc:1035] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:34.595721 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf5e8ae124786b98df6013ed28460" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:34.595852 18509 leader_election.cc:304] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:34.599020 19039 raft_consensus.cc:2798] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:34.599450 19039 raft_consensus.cc:491] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:34.599814 19039 raft_consensus.cc:3054] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.602722 18664 tablet_bootstrap.cc:492] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:34.603279 18664 ts_tablet_manager.cc:1397] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.006s
I20250112 20:58:34.605914 18664 raft_consensus.cc:357] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:34.606617 18664 raft_consensus.cc:383] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.606923 18664 raft_consensus.cc:738] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.607630 18664 consensus_queue.cc:260] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:34.609102 19039 raft_consensus.cc:513] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.609537 18664 ts_tablet_manager.cc:1428] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:58:34.610321 18662 raft_consensus.cc:383] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.610610 18664 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:34.610620 18662 raft_consensus.cc:738] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.612128 18662 consensus_queue.cc:260] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.613310 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf5e8ae124786b98df6013ed28460" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.613955 18471 raft_consensus.cc:2463] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:34.615000 18662 ts_tablet_manager.cc:1428] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20250112 20:58:34.592372 18663 raft_consensus.cc:357] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.616374 18585 leader_election.cc:304] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:34.616802 18663 raft_consensus.cc:383] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.617136 18663 raft_consensus.cc:738] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.617487 19037 raft_consensus.cc:2743] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:34.617692 18663 consensus_queue.cc:260] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.617694 18662 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:34.619421 18663 ts_tablet_manager.cc:1428] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.211s user 0.008s sys 0.000s
I20250112 20:58:34.620460 18663 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:34.621608 18661 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:34.622067 18661 ts_tablet_manager.cc:1397] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.237s user 0.011s sys 0.000s
I20250112 20:58:34.624220 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:34.624970 18545 raft_consensus.cc:2388] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2f9ce9a0415c4546b9a764ac06cefb38 in current term 1: Already voted for candidate a45f68dd3d9043809f3bb06287f5d2d3 in this term.
I20250112 20:58:34.626755 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf5e8ae124786b98df6013ed28460" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:34.627468 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 49127.3us
I20250112 20:58:34.629057 18661 raft_consensus.cc:357] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.629745 18661 raft_consensus.cc:383] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.629998 18661 raft_consensus.cc:738] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.630368 18664 tablet_bootstrap.cc:654] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.630597 18661 consensus_queue.cc:260] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.631894 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.633493 18511 leader_election.cc:304] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:34.634392 18688 raft_consensus.cc:2798] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:34.634974 18688 raft_consensus.cc:695] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:34.635557 18290 catalog_manager.cc:5526] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 1 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:34.636224 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf5e8ae124786b98df6013ed28460" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.615396 18509 leader_election.cc:304] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:34.636883 18395 raft_consensus.cc:2463] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:34.635936 18688 consensus_queue.cc:237] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.615723 18362 leader_election.cc:304] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:34.637851 19028 raft_consensus.cc:2798] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:34.638195 19028 raft_consensus.cc:491] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:34.638556 19028 raft_consensus.cc:3054] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.633646 18291 catalog_manager.cc:5526] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:34.640182 18986 raft_consensus.cc:2798] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:34.640681 18986 raft_consensus.cc:695] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:34.641428 18986 consensus_queue.cc:237] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
W20250112 20:58:34.644529 18342 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:58:34.644968 19037 raft_consensus.cc:491] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.645553 19037 raft_consensus.cc:513] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.644917 18662 tablet_bootstrap.cc:654] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.647619 19037 leader_election.cc:290] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:34.655789 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.658098 19033 consensus_queue.cc:1035] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:34.659996 18662 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:34.660445 18662 ts_tablet_manager.cc:1397] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.043s user 0.010s sys 0.000s
I20250112 20:58:34.663013 19039 leader_election.cc:290] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:34.663259 18435 leader_election.cc:304] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:34.666146 18395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.667277 18436 leader_election.cc:304] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:34.660180 19028 raft_consensus.cc:513] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.668063 18664 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:34.669363 18467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.670786 18663 tablet_bootstrap.cc:654] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.663250 18662 raft_consensus.cc:357] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.676097 18664 ts_tablet_manager.cc:1397] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.066s user 0.007s sys 0.006s
I20250112 20:58:34.677269 18662 raft_consensus.cc:383] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.677587 18662 raft_consensus.cc:738] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.679127 18436 leader_election.cc:304] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
W20250112 20:58:34.679924 18436 leader_election.cc:343] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.678876 18662 consensus_queue.cc:260] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.689968 18395 raft_consensus.cc:2388] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c60c0cbe377c450297e3d536b7f7c89c in current term 1: Already voted for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in this term.
I20250112 20:58:34.677404 19033 consensus_queue.cc:1035] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
W20250112 20:58:34.691730 18585 leader_election.cc:343] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.743906 18661 ts_tablet_manager.cc:1428] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.122s user 0.006s sys 0.001s
I20250112 20:58:34.745205 18662 ts_tablet_manager.cc:1428] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.084s user 0.007s sys 0.000s
I20250112 20:58:34.748416 19062 raft_consensus.cc:2743] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.748409 19063 raft_consensus.cc:2798] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:34.751468 19063 raft_consensus.cc:695] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:34.751611 18718 raft_consensus.cc:2798] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:34.752008 18718 raft_consensus.cc:491] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:34.752314 18718 raft_consensus.cc:3054] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.761003 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8820dfc60bc477985f861406c22dd3b" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:34.768970 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8820dfc60bc477985f861406c22dd3b" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:34.769834 18390 raft_consensus.cc:3054] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.777714 18661 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:34.744941 18585 leader_election.cc:304] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:34.786434 18664 raft_consensus.cc:357] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.789695 18664 raft_consensus.cc:383] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.790105 18664 raft_consensus.cc:738] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.790573 18662 tablet_bootstrap.cc:492] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
W20250112 20:58:34.791661 18509 leader_election.cc:343] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.794153 18664 consensus_queue.cc:260] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
W20250112 20:58:34.779649 18511 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 90012.7us
W20250112 20:58:34.797158 18361 leader_election.cc:343] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.797055 18719 consensus_queue.cc:1035] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250112 20:58:34.802012 18995 raft_consensus.cc:2743] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.791844 19039 raft_consensus.cc:491] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.752388 19063 consensus_queue.cc:237] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:34.812320 19069 log.cc:926] Time spent T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38: Append to log took a long time: real 0.057s user 0.001s sys 0.000s
I20250112 20:58:34.812724 18664 ts_tablet_manager.cc:1428] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.136s user 0.008s sys 0.002s
I20250112 20:58:34.813712 18664 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb: Bootstrap starting.
I20250112 20:58:34.811163 19039 raft_consensus.cc:513] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:34.814791 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf5e8ae124786b98df6013ed28460" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:34.800925 19028 leader_election.cc:290] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.821033 19039 leader_election.cc:290] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:34.825582 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf5e8ae124786b98df6013ed28460" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:34.826056 19037 consensus_queue.cc:1035] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250112 20:58:34.831307 18361 leader_election.cc:343] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:34.807981 18585 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 117924us
I20250112 20:58:34.834048 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:34.832000 18718 raft_consensus.cc:513] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.839922 18471 raft_consensus.cc:2388] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a45f68dd3d9043809f3bb06287f5d2d3 in current term 1: Already voted for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in this term.
I20250112 20:58:34.846983 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf5e8ae124786b98df6013ed28460" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:34.847023 18718 leader_election.cc:290] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:34.853125 18688 raft_consensus.cc:491] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.853554 18688 raft_consensus.cc:513] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.854519 18585 leader_election.cc:304] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:34.855301 18688 leader_election.cc:290] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:34.855680 19036 raft_consensus.cc:3054] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.857389 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:34.858903 19084 raft_consensus.cc:491] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.859349 19084 raft_consensus.cc:513] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.861069 19084 leader_election.cc:290] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:34.858294 18394 raft_consensus.cc:2388] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2f9ce9a0415c4546b9a764ac06cefb38 in current term 1: Already voted for candidate a45f68dd3d9043809f3bb06287f5d2d3 in this term.
W20250112 20:58:34.865005 19077 log.cc:926] Time spent T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
I20250112 20:58:34.866878 18362 leader_election.cc:304] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:34.869415 18719 raft_consensus.cc:2743] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.858099 18613 raft_consensus.cc:2463] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:34.874850 18509 leader_election.cc:304] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:34.877211 18390 raft_consensus.cc:2463] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:34.878741 18662 tablet_bootstrap.cc:654] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.879858 18664 tablet_bootstrap.cc:654] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.880787 19039 raft_consensus.cc:2798] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:34.880924 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:34.881243 19039 raft_consensus.cc:491] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:34.881585 19039 raft_consensus.cc:3054] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:58:34.893465 18509 leader_election.cc:343] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.905393 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.901132 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.909847 18511 leader_election.cc:304] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:34.924737 19085 raft_consensus.cc:2743] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.928515 18664 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb: No bootstrap required, opened a new log
I20250112 20:58:34.930991 18663 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:34.931448 18663 ts_tablet_manager.cc:1397] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.311s user 0.012s sys 0.004s
I20250112 20:58:34.929216 18664 ts_tablet_manager.cc:1397] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb: Time spent bootstrapping tablet: real 0.116s user 0.020s sys 0.006s
I20250112 20:58:34.931572 19039 raft_consensus.cc:513] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.933490 19039 leader_election.cc:290] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:34.919040 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.933938 18663 raft_consensus.cc:357] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.934608 18663 raft_consensus.cc:383] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.934896 18663 raft_consensus.cc:738] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.936743 18661 tablet_bootstrap.cc:654] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.942397 18664 raft_consensus.cc:357] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.943104 18664 raft_consensus.cc:383] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:34.943377 18664 raft_consensus.cc:738] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Initialized, Role: FOLLOWER
I20250112 20:58:34.943634 19036 raft_consensus.cc:2743] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.943926 18664 consensus_queue.cc:260] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:34.944548 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bdf5e8ae124786b98df6013ed28460" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:34.945160 18543 raft_consensus.cc:3054] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.945628 18664 ts_tablet_manager.cc:1428] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20250112 20:58:34.946396 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:34.950033 18663 consensus_queue.cc:260] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.952679 18663 ts_tablet_manager.cc:1428] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.021s user 0.007s sys 0.001s
I20250112 20:58:34.953608 18663 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:34.954774 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:34.955323 18618 raft_consensus.cc:3054] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:34.959385 18509 leader_election.cc:304] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:34.963421 18688 raft_consensus.cc:2798] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:34.964054 18688 raft_consensus.cc:695] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:34.965431 18688 consensus_queue.cc:237] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:34.967875 18995 raft_consensus.cc:491] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:34.968449 18995 raft_consensus.cc:513] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:34.971923 18995 leader_election.cc:290] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:34.976336 18663 tablet_bootstrap.cc:654] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:34.979830 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6961f63ec790480aaa86f70431ed86c6" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:34.983204 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6961f63ec790480aaa86f70431ed86c6" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:34.983866 18392 raft_consensus.cc:2463] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:34.984658 18471 raft_consensus.cc:2463] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 0.
I20250112 20:58:34.986768 18618 raft_consensus.cc:2463] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:34.989003 18585 leader_election.cc:304] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:34.989564 18543 raft_consensus.cc:2463] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
W20250112 20:58:34.990767 18509 leader_election.cc:343] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:58:34.993109 18509 leader_election.cc:343] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:34.993638 18509 leader_election.cc:304] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:34.994437 19084 raft_consensus.cc:2743] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:34.992458 19036 raft_consensus.cc:2798] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:34.999037 19036 raft_consensus.cc:491] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:35.000093 18436 leader_election.cc:304] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:35.000903 19036 raft_consensus.cc:3054] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.001030 19061 raft_consensus.cc:2798] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:35.001646 19061 raft_consensus.cc:695] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:35.002277 18662 tablet_bootstrap.cc:492] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:35.002734 18662 ts_tablet_manager.cc:1397] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.250s user 0.016s sys 0.001s
I20250112 20:58:35.002434 19061 consensus_queue.cc:237] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:35.005043 18509 leader_election.cc:343] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:35.005353 18662 raft_consensus.cc:357] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:35.006167 18662 raft_consensus.cc:383] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:35.006614 18662 raft_consensus.cc:738] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:35.008941 18662 consensus_queue.cc:260] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:35.011955 18509 leader_election.cc:304] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:35.019016 18662 ts_tablet_manager.cc:1428] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20250112 20:58:35.018411 19085 raft_consensus.cc:2798] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:35.021569 19085 raft_consensus.cc:695] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:35.021798 18661 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:35.022414 18661 ts_tablet_manager.cc:1397] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.275s user 0.022s sys 0.009s
I20250112 20:58:35.023303 18663 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:35.022938 19085 consensus_queue.cc:237] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:35.025072 18662 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38: Bootstrap starting.
I20250112 20:58:35.028861 18663 ts_tablet_manager.cc:1397] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.075s user 0.010s sys 0.003s
I20250112 20:58:35.033509 18661 raft_consensus.cc:357] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.034274 18661 raft_consensus.cc:383] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:35.034541 18661 raft_consensus.cc:738] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:35.035187 18661 consensus_queue.cc:260] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.040005 18661 ts_tablet_manager.cc:1428] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20250112 20:58:35.040908 18661 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:35.049721 19036 raft_consensus.cc:513] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.066378 18661 tablet_bootstrap.cc:654] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:35.071476 18662 tablet_bootstrap.cc:654] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:35.085106 18688 raft_consensus.cc:491] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:35.085623 18688 raft_consensus.cc:513] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:35.084429 19036 leader_election.cc:290] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:35.065284 19028 raft_consensus.cc:491] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:35.089699 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6961f63ec790480aaa86f70431ed86c6" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:35.089893 19028 raft_consensus.cc:513] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:35.090271 18471 raft_consensus.cc:3054] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.090360 18467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:35.090943 18467 raft_consensus.cc:2463] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:35.091885 18511 leader_election.cc:304] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:35.092864 19039 raft_consensus.cc:2798] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:35.093211 19039 raft_consensus.cc:491] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:35.093518 19039 raft_consensus.cc:3054] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.094103 18467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdbd6c5512664ac7a36a13bd1a89d75f" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:35.094779 18663 raft_consensus.cc:357] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.095451 18663 raft_consensus.cc:383] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:35.095710 18663 raft_consensus.cc:738] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:35.066872 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6961f63ec790480aaa86f70431ed86c6" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:35.096529 18392 raft_consensus.cc:3054] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.089721 18688 leader_election.cc:290] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:35.070377 18294 catalog_manager.cc:5526] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:35.092012 19028 leader_election.cc:290] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:35.101744 18663 consensus_queue.cc:260] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.066526 19031 raft_consensus.cc:491] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:35.104914 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdbd6c5512664ac7a36a13bd1a89d75f" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:35.111699 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:35.112357 18618 raft_consensus.cc:2463] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 0.
I20250112 20:58:35.114629 18471 raft_consensus.cc:2463] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:35.115639 18585 leader_election.cc:304] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:35.116359 19036 raft_consensus.cc:2798] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:35.125062 19085 consensus_queue.cc:1035] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:35.125389 19036 raft_consensus.cc:695] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:35.126178 19036 consensus_queue.cc:237] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.130348 18663 ts_tablet_manager.cc:1428] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.101s user 0.006s sys 0.000s
I20250112 20:58:35.131489 18663 tablet_bootstrap.cc:492] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
I20250112 20:58:35.133850 18543 raft_consensus.cc:3054] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.142879 18870 consensus_queue.cc:1035] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20250112 20:58:35.116499 19039 raft_consensus.cc:513] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:35.146545 19039 leader_election.cc:290] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:35.148571 18467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:35.149181 18467 raft_consensus.cc:3054] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.149662 18545 raft_consensus.cc:2388] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a45f68dd3d9043809f3bb06287f5d2d3 in current term 1: Already voted for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in this term.
I20250112 20:58:35.110662 19031 raft_consensus.cc:513] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.151294 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:35.151985 18616 raft_consensus.cc:3054] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.152101 18361 leader_election.cc:304] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb
I20250112 20:58:35.153122 18661 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:35.153272 19028 raft_consensus.cc:3054] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.153592 18661 ts_tablet_manager.cc:1397] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.113s user 0.031s sys 0.026s
I20250112 20:58:35.151206 18688 consensus_queue.cc:1035] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:35.162286 19031 leader_election.cc:290] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:35.176945 18661 raft_consensus.cc:357] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.177634 18661 raft_consensus.cc:383] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:35.177867 18661 raft_consensus.cc:738] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:35.178493 18661 consensus_queue.cc:260] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.179164 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:35.179770 18618 raft_consensus.cc:2463] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:35.180620 18661 ts_tablet_manager.cc:1428] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.027s user 0.005s sys 0.001s
I20250112 20:58:35.181946 18392 raft_consensus.cc:2463] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:35.183302 18663 tablet_bootstrap.cc:654] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:35.184074 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:35.169428 18870 raft_consensus.cc:491] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:35.185683 18870 raft_consensus.cc:513] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.186406 18616 raft_consensus.cc:2463] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:35.187541 18870 leader_election.cc:290] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:35.187567 18509 leader_election.cc:304] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:35.188514 18688 raft_consensus.cc:2798] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:35.188932 18688 raft_consensus.cc:695] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:35.183255 18360 leader_election.cc:304] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:35.189574 18688 consensus_queue.cc:237] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
W20250112 20:58:35.191782 18362 leader_election.cc:343] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:35.192553 19028 raft_consensus.cc:2743] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:35.193610 19007 raft_consensus.cc:2798] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:35.193969 19007 raft_consensus.cc:491] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:35.194253 19007 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.198331 18661 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:35.208209 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:35.208786 19098 raft_consensus.cc:491] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:35.208884 18613 raft_consensus.cc:2463] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:35.210490 18824 consensus_queue.cc:1035] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:35.211005 18435 leader_election.cc:304] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:35.205472 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:35.226202 18392 raft_consensus.cc:2463] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 0.
I20250112 20:58:35.218451 18870 raft_consensus.cc:2798] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:35.209321 19098 raft_consensus.cc:513] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:35.208169 18467 raft_consensus.cc:2463] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:35.241281 19098 leader_election.cc:290] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:35.226962 18870 raft_consensus.cc:491] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:35.253661 18394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8820dfc60bc477985f861406c22dd3b" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:35.254503 18394 raft_consensus.cc:2388] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c60c0cbe377c450297e3d536b7f7c89c in current term 1: Already voted for candidate 88ef62b247304e079788e24ddfdaa4fb in this term.
I20250112 20:58:35.257403 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8820dfc60bc477985f861406c22dd3b" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:35.261993 18583 leader_election.cc:304] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:35.253099 18870 raft_consensus.cc:3054] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.265765 19061 consensus_queue.cc:1035] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:35.274391 18661 tablet_bootstrap.cc:654] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:35.278009 18870 raft_consensus.cc:513] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.282341 18662 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38: No bootstrap required, opened a new log
I20250112 20:58:35.282071 18688 consensus_queue.cc:1035] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:35.284008 18662 ts_tablet_manager.cc:1397] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent bootstrapping tablet: real 0.259s user 0.019s sys 0.013s
I20250112 20:58:35.281419 19036 raft_consensus.cc:3054] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.292623 18870 leader_election.cc:290] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:35.293291 18616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:35.293920 18616 raft_consensus.cc:3054] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.279559 19007 raft_consensus.cc:513] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.298733 18392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
I20250112 20:58:35.300794 19039 consensus_queue.cc:1035] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:35.301115 18392 raft_consensus.cc:3054] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.307538 18662 raft_consensus.cc:357] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.308431 18662 raft_consensus.cc:383] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:35.308706 18662 raft_consensus.cc:738] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Initialized, Role: FOLLOWER
I20250112 20:58:35.310009 18662 consensus_queue.cc:260] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.305425 18870 consensus_queue.cc:1035] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20250112 20:58:35.325271 19007 leader_election.cc:290] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:35.327010 18612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:35.326907 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:35.327803 18612 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.327955 18470 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.341012 18661 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:35.341789 18661 ts_tablet_manager.cc:1397] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.160s user 0.011s sys 0.008s
I20250112 20:58:35.356387 18995 raft_consensus.cc:491] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:35.358062 18995 raft_consensus.cc:513] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.360123 18995 leader_election.cc:290] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:35.366781 18467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:35.332617 18616 raft_consensus.cc:2463] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:35.369310 18435 leader_election.cc:304] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:35.371552 18718 raft_consensus.cc:2798] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:35.371953 18718 raft_consensus.cc:695] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:35.372721 18718 consensus_queue.cc:237] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.373844 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:35.375422 18293 catalog_manager.cc:5526] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } }
I20250112 20:58:35.378051 18619 raft_consensus.cc:3054] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.381232 19036 raft_consensus.cc:2743] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:35.385376 18662 ts_tablet_manager.cc:1428] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38: Time spent starting tablet: real 0.101s user 0.006s sys 0.000s
I20250112 20:58:35.388336 18661 raft_consensus.cc:357] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:35.393057 18585 leader_election.cc:304] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:35.393517 18661 raft_consensus.cc:383] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:35.394181 18661 raft_consensus.cc:738] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:35.395401 18661 consensus_queue.cc:260] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:35.404757 18661 ts_tablet_manager.cc:1428] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.063s user 0.007s sys 0.001s
I20250112 20:58:35.411317 18661 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:35.422214 18417 debug-util.cc:398] Leaking SignalData structure 0x7b080035a5c0 after lost signal to thread 15457
W20250112 20:58:35.423355 18417 debug-util.cc:398] Leaking SignalData structure 0x7b080015b720 after lost signal to thread 18328
W20250112 20:58:35.424553 18417 debug-util.cc:398] Leaking SignalData structure 0x7b08002214c0 after lost signal to thread 18420
W20250112 20:58:35.425621 18417 debug-util.cc:398] Leaking SignalData structure 0x7b0800245020 after lost signal to thread 18495
W20250112 20:58:35.427677 18417 debug-util.cc:398] Leaking SignalData structure 0x7b08002929e0 after lost signal to thread 18569
W20250112 20:58:35.428794 18417 debug-util.cc:398] Leaking SignalData structure 0x7b080004e0c0 after lost signal to thread 18643
W20250112 20:58:35.429687 18417 debug-util.cc:398] Leaking SignalData structure 0x7b0800303520 after lost signal to thread 19030
I20250112 20:58:35.433087 18291 catalog_manager.cc:5526] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 0 to 1, leader changed from <none> to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 1 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:35.449693 18294 catalog_manager.cc:5526] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:35.457440 18294 catalog_manager.cc:5526] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
W20250112 20:58:35.453641 19112 log.cc:926] Time spent T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb: Append to log took a long time: real 0.100s user 0.000s sys 0.000s
I20250112 20:58:35.469584 18294 catalog_manager.cc:5526] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
W20250112 20:58:35.501802 18509 consensus_peers.cc:487] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:35.502395 18470 raft_consensus.cc:2463] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:35.513883 18880 consensus_queue.cc:1035] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:35.520624 18661 tablet_bootstrap.cc:654] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:35.524641 19044 consensus_queue.cc:1035] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:35.559361 18362 leader_election.cc:304] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:35.560120 18719 raft_consensus.cc:2798] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:58:35.560571 18719 raft_consensus.cc:695] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:35.561295 18719 consensus_queue.cc:237] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:35.592262 18663 tablet_bootstrap.cc:492] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:35.592916 18663 ts_tablet_manager.cc:1397] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.462s user 0.012s sys 0.021s
W20250112 20:58:35.590373 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 101377us
I20250112 20:58:35.590849 19028 raft_consensus.cc:491] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:35.607079 19028 raft_consensus.cc:513] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:35.610282 19028 leader_election.cc:290] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:35.594578 18612 raft_consensus.cc:2463] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
W20250112 20:58:35.643146 19115 log.cc:926] Time spent T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20250112 20:58:35.568434 19044 consensus_queue.cc:1035] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20250112 20:58:35.593552 18392 raft_consensus.cc:2463] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:35.663888 18538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:35.675606 19080 log.cc:926] Time spent T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
I20250112 20:58:35.698504 18612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:35.704929 18995 raft_consensus.cc:2743] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:35.714824 19063 consensus_queue.cc:1035] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250112 20:58:35.716742 18342 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:58:35.720945 19118 consensus_queue.cc:1035] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250112 20:58:35.725914 18661 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: No bootstrap required, opened a new log
I20250112 20:58:35.726378 18661 ts_tablet_manager.cc:1397] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.315s user 0.013s sys 0.003s
I20250112 20:58:35.718859 18663 raft_consensus.cc:357] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:35.741077 18663 raft_consensus.cc:383] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:35.741461 18663 raft_consensus.cc:738] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:35.742249 18663 consensus_queue.cc:260] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:35.691143 18538 raft_consensus.cc:2463] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 0.
I20250112 20:58:35.765997 18293 catalog_manager.cc:5526] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } }
I20250112 20:58:35.770805 18688 consensus_queue.cc:1035] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:35.772073 18663 ts_tablet_manager.cc:1428] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.179s user 0.007s sys 0.000s
W20250112 20:58:35.808570 18360 leader_election.cc:343] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:35.824283 18663 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c: Bootstrap starting.
W20250112 20:58:35.828989 19119 log.cc:926] Time spent T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38: Append to log took a long time: real 0.056s user 0.002s sys 0.000s
I20250112 20:58:35.816789 18995 consensus_queue.cc:1035] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250112 20:58:35.832891 18661 raft_consensus.cc:357] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:35.835503 18661 raft_consensus.cc:383] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:35.835682 18661 raft_consensus.cc:738] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:35.836037 18661 consensus_queue.cc:260] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
W20250112 20:58:35.851310 18509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 74675.8us
I20250112 20:58:35.869715 18661 ts_tablet_manager.cc:1428] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.143s user 0.007s sys 0.000s
W20250112 20:58:35.886762 19120 log.cc:926] Time spent T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c: Append to log took a long time: real 0.166s user 0.000s sys 0.001s
I20250112 20:58:35.876358 18361 leader_election.cc:304] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters: c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:35.892122 18824 consensus_queue.cc:1035] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:35.893679 19007 raft_consensus.cc:2798] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:58:35.894055 19007 raft_consensus.cc:491] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:35.894332 19007 raft_consensus.cc:3054] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:35.913722 18290 catalog_manager.cc:5526] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:35.927361 18663 tablet_bootstrap.cc:654] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c: Neither blocks nor log segments found. Creating new log.
I20250112 20:58:35.981060 18718 consensus_queue.cc:1035] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:35.973398 19084 consensus_queue.cc:1035] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20250112 20:58:36.029659 18718 consensus_queue.cc:1035] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:36.050806 18291 catalog_manager.cc:5526] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 1 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:36.098659 19139 raft_consensus.cc:491] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:36.126405 18613 raft_consensus.cc:3054] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:36.151249 19007 raft_consensus.cc:513] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:36.118256 19139 raft_consensus.cc:513] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:36.162973 19031 consensus_queue.cc:1035] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:58:36.176383 18294 catalog_manager.cc:5526] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:36.181854 19007 leader_election.cc:290] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:36.189384 18663 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c: No bootstrap required, opened a new log
I20250112 20:58:36.189869 18663 ts_tablet_manager.cc:1397] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c: Time spent bootstrapping tablet: real 0.366s user 0.019s sys 0.023s
I20250112 20:58:36.191779 19139 leader_election.cc:290] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:36.159852 18391 raft_consensus.cc:3054] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:36.196383 18719 consensus_queue.cc:1035] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:36.209264 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:36.209939 18612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:36.213009 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:36.214608 18663 raft_consensus.cc:357] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:36.215526 18663 raft_consensus.cc:383] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:58:36.215837 18663 raft_consensus.cc:738] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Initialized, Role: FOLLOWER
I20250112 20:58:36.217300 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:36.217883 18543 raft_consensus.cc:3054] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:36.220114 18361 leader_election.cc:304] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3; no voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:36.227393 18543 raft_consensus.cc:2463] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 1.
I20250112 20:58:36.228642 18361 leader_election.cc:304] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:36.229686 18663 consensus_queue.cc:260] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:36.230234 19031 raft_consensus.cc:2798] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader election won for term 1
W20250112 20:58:36.230665 18360 leader_election.cc:343] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:58:36.230716 19031 raft_consensus.cc:695] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:36.234220 19031 consensus_queue.cc:237] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:36.247529 18290 catalog_manager.cc:5526] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:36.266580 18663 ts_tablet_manager.cc:1428] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c: Time spent starting tablet: real 0.076s user 0.006s sys 0.002s
I20250112 20:58:36.313812 19140 raft_consensus.cc:2743] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:36.357367 18293 catalog_manager.cc:5526] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:36.528044 19137 raft_consensus.cc:491] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:36.528470 19137 raft_consensus.cc:513] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:36.530441 19137 leader_election.cc:290] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:36.531446 18391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" is_pre_election: true
I20250112 20:58:36.531849 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:36.532711 18541 raft_consensus.cc:2388] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c60c0cbe377c450297e3d536b7f7c89c in current term 1: Already voted for candidate a45f68dd3d9043809f3bb06287f5d2d3 in this term.
I20250112 20:58:36.534292 18583 leader_election.cc:304] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:36.535725 19137 raft_consensus.cc:3054] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:58:36.542845 19137 raft_consensus.cc:2743] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:58:36.643908 18719 consensus_queue.cc:1035] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:36.653795 19140 consensus_queue.cc:1035] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:58:36.933145 15454 tablet_server.cc:178] TabletServer@127.15.23.129:0 shutting down...
W20250112 20:58:36.945248 18583 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 206 similar messages]
W20250112 20:58:36.975497 18583 consensus_peers.cc:487] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.026968 18509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 53757.1us
W20250112 20:58:37.030849 18509 consensus_peers.cc:487] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.033923 18509 consensus_peers.cc:487] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.034971 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
W20250112 20:58:37.033008 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:38493, user_credentials={real_user=slave}} blocked reactor thread for 63433.5us
I20250112 20:58:37.037034 15454 tablet_replica.cc:331] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.037956 15454 raft_consensus.cc:2238] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
W20250112 20:58:37.039783 18435 consensus_peers.cc:487] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.040112 15454 raft_consensus.cc:2267] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:37.043602 18435 consensus_peers.cc:487] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.044747 15454 tablet_replica.cc:331] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.045609 15454 raft_consensus.cc:2238] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.046088 15454 raft_consensus.cc:2267] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.049367 15454 tablet_replica.cc:331] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.050249 15454 raft_consensus.cc:2238] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.050671 15454 raft_consensus.cc:2267] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.053599 15454 tablet_replica.cc:331] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.054116 15454 raft_consensus.cc:2238] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.054481 15454 raft_consensus.cc:2267] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.056385 15454 tablet_replica.cc:331] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.057082 15454 raft_consensus.cc:2238] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.057435 15454 raft_consensus.cc:2267] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.059226 15454 tablet_replica.cc:331] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
W20250112 20:58:37.059248 18509 consensus_peers.cc:487] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.059810 15454 raft_consensus.cc:2238] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.060204 15454 raft_consensus.cc:2267] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.062018 15454 tablet_replica.cc:331] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.062548 15454 raft_consensus.cc:2238] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.063360 15454 raft_consensus.cc:2267] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.065562 15454 tablet_replica.cc:331] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.066305 15454 raft_consensus.cc:2238] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.066728 15454 raft_consensus.cc:2267] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.068622 15454 tablet_replica.cc:331] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.069211 15454 raft_consensus.cc:2238] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.069885 15454 raft_consensus.cc:2267] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.071604 15454 tablet_replica.cc:331] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.072173 15454 raft_consensus.cc:2238] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.072563 15454 raft_consensus.cc:2267] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.074319 15454 tablet_replica.cc:331] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.074782 15454 raft_consensus.cc:2238] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.075127 15454 raft_consensus.cc:2267] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.076750 15454 tablet_replica.cc:331] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.077188 15454 raft_consensus.cc:2238] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.077793 15454 raft_consensus.cc:2267] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.079643 15454 tablet_replica.cc:331] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.080174 15454 raft_consensus.cc:2238] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.080978 15454 raft_consensus.cc:2267] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.083046 15454 tablet_replica.cc:331] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.083632 15454 raft_consensus.cc:2238] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.083972 15454 raft_consensus.cc:2267] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.085628 15454 tablet_replica.cc:331] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.086110 15454 raft_consensus.cc:2238] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.086774 15454 raft_consensus.cc:2267] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.088537 15454 tablet_replica.cc:331] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.089071 15454 raft_consensus.cc:2238] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.090230 15454 raft_consensus.cc:2267] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.092495 15454 tablet_replica.cc:331] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.093026 15454 raft_consensus.cc:2238] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.093375 15454 raft_consensus.cc:2267] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.095220 15454 tablet_replica.cc:331] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.095956 15454 raft_consensus.cc:2238] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.096432 15454 raft_consensus.cc:2267] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.098181 15454 tablet_replica.cc:331] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.098701 15454 raft_consensus.cc:2238] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.099041 15454 raft_consensus.cc:2267] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.101097 15454 tablet_replica.cc:331] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.101693 15454 raft_consensus.cc:2238] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.102047 15454 raft_consensus.cc:2267] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.103711 15454 tablet_replica.cc:331] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.104331 15454 raft_consensus.cc:2238] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.104727 15454 raft_consensus.cc:2267] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.107896 15454 tablet_replica.cc:331] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.108893 15454 raft_consensus.cc:2238] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.109892 15454 raft_consensus.cc:2267] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.112485 15454 tablet_replica.cc:331] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.113391 15454 raft_consensus.cc:2238] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.114418 15454 raft_consensus.cc:2267] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.116813 15454 tablet_replica.cc:331] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.117552 15454 raft_consensus.cc:2238] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.118585 15454 raft_consensus.cc:2267] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.121199 15454 tablet_replica.cc:331] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.121963 15454 raft_consensus.cc:2238] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.122519 15454 raft_consensus.cc:2267] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:37.124783 18435 consensus_peers.cc:487] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.125142 15454 tablet_replica.cc:331] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.126178 15454 raft_consensus.cc:2238] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.126735 15454 raft_consensus.cc:2267] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.129292 15454 tablet_replica.cc:331] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.130051 15454 raft_consensus.cc:2238] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:37.130484 18583 consensus_peers.cc:487] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.130599 15454 raft_consensus.cc:2267] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:37.130733 18435 consensus_peers.cc:487] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.133183 15454 tablet_replica.cc:331] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.133970 15454 raft_consensus.cc:2238] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.134490 15454 raft_consensus.cc:2267] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.136970 15454 tablet_replica.cc:331] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
W20250112 20:58:37.137293 18509 consensus_peers.cc:487] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.137799 15454 raft_consensus.cc:2238] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.138867 15454 raft_consensus.cc:2267] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.141538 15454 tablet_replica.cc:331] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
W20250112 20:58:37.142094 18435 consensus_peers.cc:487] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.142366 15454 raft_consensus.cc:2238] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.143536 15454 raft_consensus.cc:2267] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.146498 15454 tablet_replica.cc:331] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.147315 15454 raft_consensus.cc:2238] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.147873 15454 raft_consensus.cc:2267] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.150429 15454 tablet_replica.cc:331] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
W20250112 20:58:37.151196 18583 consensus_peers.cc:487] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.151254 15454 raft_consensus.cc:2238] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.151968 15454 raft_consensus.cc:2267] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.154475 15454 tablet_replica.cc:331] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.155231 15454 raft_consensus.cc:2238] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.155699 15454 raft_consensus.cc:2267] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.158320 15454 tablet_replica.cc:331] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.159107 15454 raft_consensus.cc:2238] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.159619 15454 raft_consensus.cc:2267] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:37.160423 18435 consensus_peers.cc:487] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.162604 15454 tablet_replica.cc:331] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.163471 15454 raft_consensus.cc:2238] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:37.163720 18435 consensus_peers.cc:487] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.164021 15454 raft_consensus.cc:2267] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:37.165654 18435 consensus_peers.cc:487] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.166828 15454 tablet_replica.cc:331] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.167631 15454 raft_consensus.cc:2238] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.168184 15454 raft_consensus.cc:2267] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.170553 15454 tablet_replica.cc:331] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.171301 15454 raft_consensus.cc:2238] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.171794 15454 raft_consensus.cc:2267] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.174341 15454 tablet_replica.cc:331] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.175060 15454 raft_consensus.cc:2238] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.176021 15454 raft_consensus.cc:2267] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.178632 15454 tablet_replica.cc:331] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.179364 15454 raft_consensus.cc:2238] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.179939 15454 raft_consensus.cc:2267] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:37.182086 18435 consensus_peers.cc:487] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.182608 15454 tablet_replica.cc:331] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.183400 15454 raft_consensus.cc:2238] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.184335 15454 raft_consensus.cc:2267] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.186915 15454 tablet_replica.cc:331] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.187610 15454 raft_consensus.cc:2238] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.188105 15454 raft_consensus.cc:2267] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.190848 15454 tablet_replica.cc:331] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.191624 15454 raft_consensus.cc:2238] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:37.192646 15454 raft_consensus.cc:2267] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.195238 15454 tablet_replica.cc:331] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.195968 15454 raft_consensus.cc:2238] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.196511 15454 raft_consensus.cc:2267] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:37.197263 18435 consensus_peers.cc:487] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:37.199156 15454 tablet_replica.cc:331] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.199923 15454 raft_consensus.cc:2238] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.200495 15454 raft_consensus.cc:2267] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:37.203227 15454 tablet_replica.cc:331] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: stopping tablet replica
I20250112 20:58:37.203860 15454 raft_consensus.cc:2238] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:37.204378 15454 raft_consensus.cc:2267] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:37.230612 18435 consensus_peers.cc:487] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.251653 18509 consensus_peers.cc:487] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.293452 18509 consensus_peers.cc:487] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.293812 18509 consensus_peers.cc:487] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.301391 18435 consensus_peers.cc:487] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.341715 18509 consensus_peers.cc:487] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.350582 18435 consensus_peers.cc:487] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.350983 18435 consensus_peers.cc:487] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.357088 18509 consensus_peers.cc:487] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.369590 18435 consensus_peers.cc:487] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.388259 18583 consensus_peers.cc:487] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.494280 18509 consensus_peers.cc:487] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:37.537659 18435 consensus_peers.cc:487] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:38.022488 19167 raft_consensus.cc:491] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.023034 19167 raft_consensus.cc:513] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.025190 19167 leader_election.cc:290] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.026553 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:38.028492 18509 leader_election.cc:336] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.028856 18509 leader_election.cc:304] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:38.029343 19167 raft_consensus.cc:2743] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.089937 19137 raft_consensus.cc:491] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.090510 19137 raft_consensus.cc:513] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.092432 19137 leader_election.cc:290] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.094300 19137 raft_consensus.cc:491] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.095187 19137 raft_consensus.cc:513] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.095496 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:38.098874 18583 leader_election.cc:336] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.099318 18583 leader_election.cc:304] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.100515 18995 raft_consensus.cc:2743] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.101372 19137 leader_election.cc:290] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.102563 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59cd634f5244017b18a111892243f57" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
W20250112 20:58:38.106559 18583 leader_election.cc:336] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.106869 18583 leader_election.cc:304] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.107374 19137 raft_consensus.cc:2743] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.153240 19167 raft_consensus.cc:491] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.153906 19167 raft_consensus.cc:513] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.155972 19167 leader_election.cc:290] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.156641 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:38.157341 18615 raft_consensus.cc:2463] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:38.158263 18509 leader_election.cc:304] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:38.158820 19167 raft_consensus.cc:2798] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.159262 19167 raft_consensus.cc:491] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.159648 19167 raft_consensus.cc:3054] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:38.159898 18509 leader_election.cc:336] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.166898 19167 raft_consensus.cc:513] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.168895 19167 leader_election.cc:290] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.171242 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc3d5bd7d04459c9c1e7ccb28eb2ee7" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:38.171747 18615 raft_consensus.cc:3054] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.172253 19017 raft_consensus.cc:491] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.172794 19017 raft_consensus.cc:513] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
W20250112 20:58:38.173718 18509 leader_election.cc:336] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.175192 19017 leader_election.cc:290] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:38.176899 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:38.177920 18435 leader_election.cc:336] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.178318 19137 raft_consensus.cc:491] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.178721 18436 leader_election.cc:304] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.178791 19137 raft_consensus.cc:513] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.180549 19017 raft_consensus.cc:2743] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.181218 19167 raft_consensus.cc:491] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.181300 19137 leader_election.cc:290] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.181859 19167 raft_consensus.cc:513] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.182523 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.184027 19167 leader_election.cc:290] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.185302 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.186058 18471 raft_consensus.cc:2463] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:38.187070 18511 leader_election.cc:304] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
W20250112 20:58:38.190644 18583 leader_election.cc:336] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.191180 18583 leader_election.cc:304] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
W20250112 20:58:38.191493 18509 leader_election.cc:336] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.191972 19137 raft_consensus.cc:2743] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.193766 18615 raft_consensus.cc:2463] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 2.
I20250112 20:58:38.197959 18509 leader_election.cc:304] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.198921 19160 raft_consensus.cc:2798] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.199276 19160 raft_consensus.cc:491] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.199676 19160 raft_consensus.cc:3054] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.200341 19167 raft_consensus.cc:2798] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.200660 19017 raft_consensus.cc:491] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.200971 19167 raft_consensus.cc:695] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:38.201875 19167 consensus_queue.cc:237] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.201097 19017 raft_consensus.cc:513] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.204547 19017 leader_election.cc:290] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.206571 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:38.209261 19160 raft_consensus.cc:513] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.221781 19160 leader_election.cc:290] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
W20250112 20:58:38.219036 18435 leader_election.cc:336] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.222383 18435 leader_election.cc:304] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:38.223080 19017 raft_consensus.cc:2743] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.226253 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe68bc3c3a54ebb816b85651cfd9d08" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:38.226876 18470 raft_consensus.cc:3054] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:38.228650 18509 leader_election.cc:336] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.233336 18470 raft_consensus.cc:2463] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 2.
I20250112 20:58:38.233975 18290 catalog_manager.cc:5526] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 2 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:38.235247 18511 leader_election.cc:304] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.236781 19167 raft_consensus.cc:2798] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.237531 19167 raft_consensus.cc:695] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:38.238430 19167 consensus_queue.cc:237] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.263466 18290 catalog_manager.cc:5526] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 2 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } }
W20250112 20:58:38.286608 15454 thread.cc:535] Waited for 1000ms trying to join with diag-logger (tid 18417)
I20250112 20:58:38.290021 19017 raft_consensus.cc:491] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.290444 19017 raft_consensus.cc:513] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.292076 19017 leader_election.cc:290] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.292712 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:38.296278 18435 leader_election.cc:336] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.296676 18435 leader_election.cc:304] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:38.297178 19017 raft_consensus.cc:2743] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.315428 19137 raft_consensus.cc:491] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.315945 19137 raft_consensus.cc:513] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.318606 19137 leader_election.cc:290] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
W20250112 20:58:38.321411 18583 leader_election.cc:336] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.322353 19137 raft_consensus.cc:491] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.322513 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.322986 19137 raft_consensus.cc:513] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.323732 18585 leader_election.cc:304] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.324383 18995 raft_consensus.cc:2743] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.325430 19137 leader_election.cc:290] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.327010 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.327600 18470 raft_consensus.cc:2463] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:38.328439 18585 leader_election.cc:304] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
W20250112 20:58:38.329062 18583 leader_election.cc:336] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.329234 19137 raft_consensus.cc:2798] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.329685 19137 raft_consensus.cc:491] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.330044 19137 raft_consensus.cc:3054] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.337705 19017 raft_consensus.cc:491] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.337534 19137 raft_consensus.cc:513] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.338112 19017 raft_consensus.cc:513] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.339514 19017 leader_election.cc:290] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.340164 19137 leader_election.cc:290] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.340322 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa320ac4d374bcc8a1af58dc71bcdfe" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:38.340968 18470 raft_consensus.cc:3054] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.343461 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:38.343814 18583 leader_election.cc:336] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.344038 18619 raft_consensus.cc:2463] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:38.344761 19017 raft_consensus.cc:491] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.345184 19017 raft_consensus.cc:513] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.346766 19017 leader_election.cc:290] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:38.347827 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:38.349797 18435 leader_election.cc:304] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:38.350453 19017 raft_consensus.cc:2798] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.351387 19017 raft_consensus.cc:491] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.351776 19017 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:38.360991 18435 leader_election.cc:336] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
W20250112 20:58:38.361728 18435 leader_election.cc:336] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.362087 18435 leader_election.cc:304] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.362912 18718 raft_consensus.cc:2743] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.364538 18470 raft_consensus.cc:2463] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:38.364771 19017 raft_consensus.cc:513] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.365497 18585 leader_election.cc:304] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.366086 19137 raft_consensus.cc:2798] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.366542 19137 raft_consensus.cc:695] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:38.366593 19017 leader_election.cc:290] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.367404 19137 consensus_queue.cc:237] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.368036 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:38.369963 18619 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:38.386107 18435 leader_election.cc:336] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.387108 18619 raft_consensus.cc:2463] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 2.
I20250112 20:58:38.390014 18435 leader_election.cc:304] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.391065 19017 raft_consensus.cc:2798] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.391530 19017 raft_consensus.cc:695] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:38.391738 19167 raft_consensus.cc:491] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.392259 19017 consensus_queue.cc:237] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.395301 18290 catalog_manager.cc:5526] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:38.392279 19167 raft_consensus.cc:513] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.400660 19167 leader_election.cc:290] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.402730 19165 raft_consensus.cc:491] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.403280 19165 raft_consensus.cc:513] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.405313 19165 leader_election.cc:290] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.406041 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.406705 18471 raft_consensus.cc:2463] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:38.407858 18511 leader_election.cc:304] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
W20250112 20:58:38.408632 18509 leader_election.cc:336] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.408685 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:38.409499 18615 raft_consensus.cc:2463] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:38.409889 19160 raft_consensus.cc:2798] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.410288 19160 raft_consensus.cc:491] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.410691 19160 raft_consensus.cc:3054] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.418087 18290 catalog_manager.cc:5526] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:38.421624 19137 raft_consensus.cc:491] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.422251 19137 raft_consensus.cc:513] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.423707 19160 raft_consensus.cc:513] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.425000 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:38.425175 19137 leader_election.cc:290] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.427601 18435 leader_election.cc:304] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:38.428396 19017 raft_consensus.cc:2798] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.428776 19017 raft_consensus.cc:491] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.429116 19017 raft_consensus.cc:3054] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:38.432451 18583 leader_election.cc:336] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.432971 18583 leader_election.cc:304] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.433516 18718 raft_consensus.cc:491] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.434284 18718 raft_consensus.cc:513] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.435562 18718 leader_election.cc:290] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:38.435750 19017 raft_consensus.cc:513] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.436697 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
W20250112 20:58:38.439005 18435 leader_election.cc:336] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.439502 19137 raft_consensus.cc:2743] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250112 20:58:38.442476 18435 leader_election.cc:336] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
W20250112 20:58:38.443176 18509 leader_election.cc:336] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.443100 18435 leader_election.cc:304] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.445439 19165 raft_consensus.cc:2743] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.446138 19160 leader_election.cc:290] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.446905 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b6dc91797f14584b833e9dec80369f5" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:38.447500 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7f1fd690e5d4207957654f28756dfaf" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:38.447863 18470 raft_consensus.cc:3054] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.448133 18615 raft_consensus.cc:3054] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.444543 19017 leader_election.cc:290] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
W20250112 20:58:38.453790 18435 leader_election.cc:336] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.453998 18470 raft_consensus.cc:2463] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 2.
I20250112 20:58:38.455085 18511 leader_election.cc:304] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.455698 19160 raft_consensus.cc:2798] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.456122 19160 raft_consensus.cc:695] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:38.456858 19160 consensus_queue.cc:237] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.457950 18615 raft_consensus.cc:2463] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 2.
I20250112 20:58:38.459228 18435 leader_election.cc:304] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.460063 19017 raft_consensus.cc:2798] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.461360 19017 raft_consensus.cc:695] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:38.462069 19017 consensus_queue.cc:237] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.473425 19165 raft_consensus.cc:491] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.473937 19165 raft_consensus.cc:513] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.474323 18290 catalog_manager.cc:5526] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 2 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:38.475734 19165 leader_election.cc:290] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.479243 18291 catalog_manager.cc:5526] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:38.479835 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
W20250112 20:58:38.487553 18435 leader_election.cc:336] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.488520 18435 leader_election.cc:304] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:38.490483 18718 raft_consensus.cc:2743] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.505339 19017 raft_consensus.cc:491] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.505713 19017 raft_consensus.cc:513] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.507268 19017 leader_election.cc:290] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.507799 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:38.508342 18615 raft_consensus.cc:2463] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:38.510543 18435 leader_election.cc:304] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:38.511317 19017 raft_consensus.cc:2798] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.511579 19017 raft_consensus.cc:491] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.511811 19017 raft_consensus.cc:3054] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:38.513001 18435 leader_election.cc:336] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.517462 19137 raft_consensus.cc:491] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.517988 19137 raft_consensus.cc:513] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.519881 19137 leader_election.cc:290] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.520628 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:38.521344 18541 raft_consensus.cc:2463] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:38.522714 18584 leader_election.cc:304] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:38.523555 18995 raft_consensus.cc:491] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
W20250112 20:58:38.524133 18583 leader_election.cc:336] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.524004 18995 raft_consensus.cc:513] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.525017 19137 raft_consensus.cc:2798] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.525393 19137 raft_consensus.cc:491] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.525734 19137 raft_consensus.cc:3054] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.525897 18995 leader_election.cc:290] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.527753 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.527916 18995 raft_consensus.cc:491] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.528364 18470 raft_consensus.cc:2463] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:38.529457 18585 leader_election.cc:304] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
W20250112 20:58:38.531076 18583 leader_election.cc:336] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.528376 18995 raft_consensus.cc:513] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.538115 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:38.540078 18995 leader_election.cc:290] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.519444 19017 raft_consensus.cc:513] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.540766 18995 raft_consensus.cc:2798] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.541178 18995 raft_consensus.cc:491] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.541541 18995 raft_consensus.cc:3054] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.541837 19017 leader_election.cc:290] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.543969 19137 raft_consensus.cc:513] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.544780 19160 raft_consensus.cc:491] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.547719 19160 raft_consensus.cc:513] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
W20250112 20:58:38.547078 18435 leader_election.cc:336] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.550065 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db962a176db442490ad2e6133babb67" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:38.550765 18541 raft_consensus.cc:3054] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.550217 19137 leader_election.cc:290] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.552255 19160 leader_election.cc:290] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.563227 19136 raft_consensus.cc:491] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.563745 19136 raft_consensus.cc:513] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.565790 19136 leader_election.cc:290] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.567440 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.568096 18470 raft_consensus.cc:2463] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:38.568275 18541 raft_consensus.cc:2463] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:38.569161 18511 leader_election.cc:304] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:38.569487 18584 leader_election.cc:304] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:38.570045 19136 raft_consensus.cc:2798] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.570423 19136 raft_consensus.cc:491] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.570446 19137 raft_consensus.cc:2798] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.570900 19136 raft_consensus.cc:3054] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.571125 19137 raft_consensus.cc:695] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:38.572120 19137 consensus_queue.cc:237] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
W20250112 20:58:38.585036 18509 leader_election.cc:336] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.597692 18291 catalog_manager.cc:5526] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:38.579509 19136 raft_consensus.cc:513] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.592046 19160 raft_consensus.cc:491] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.603255 19160 raft_consensus.cc:513] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.605494 19160 leader_election.cc:290] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.607259 19136 leader_election.cc:290] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
W20250112 20:58:38.607661 18509 leader_election.cc:336] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.608899 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e59ecf931f041feba70ee5460de09da" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:38.609521 18470 raft_consensus.cc:3054] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.616081 18470 raft_consensus.cc:2463] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 2.
I20250112 20:58:38.617049 18511 leader_election.cc:304] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:38.618647 18995 raft_consensus.cc:513] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.617961 19136 raft_consensus.cc:2798] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.621069 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:38.621493 19136 raft_consensus.cc:695] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:38.621760 18615 raft_consensus.cc:2463] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:38.620345 19179 raft_consensus.cc:491] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.622352 19136 consensus_queue.cc:237] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.622848 19179 raft_consensus.cc:513] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.624028 18995 leader_election.cc:290] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.624946 19179 leader_election.cc:290] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.628705 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:38.629426 18615 raft_consensus.cc:2463] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
W20250112 20:58:38.631913 18509 leader_election.cc:336] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
W20250112 20:58:38.632625 18509 leader_election.cc:336] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
W20250112 20:58:38.632927 18585 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 103580us
I20250112 20:58:38.633585 18509 leader_election.cc:304] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.634197 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6df407527b0d446795e620dec471e027" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
W20250112 20:58:38.633699 18583 leader_election.cc:336] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.634714 18470 raft_consensus.cc:3054] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.634862 18583 leader_election.cc:304] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.635535 19160 raft_consensus.cc:2798] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.635886 19160 raft_consensus.cc:491] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.636199 19160 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.638247 19085 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.638756 19085 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.640575 19085 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.642761 19160 raft_consensus.cc:513] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.646811 19017 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.647321 19017 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.648315 19160 leader_election.cc:290] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.644243 19179 raft_consensus.cc:2743] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.649242 19165 raft_consensus.cc:491] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
W20250112 20:58:38.644569 18583 leader_election.cc:336] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.649775 19165 raft_consensus.cc:513] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.643332 18509 leader_election.cc:304] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.651670 19165 leader_election.cc:290] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.651506 18291 catalog_manager.cc:5526] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 2 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:38.652441 19160 raft_consensus.cc:2798] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.652829 19160 raft_consensus.cc:491] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.653239 19017 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:38.645800 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.654137 18471 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 1.
I20250112 20:58:38.655372 18511 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:38.655956 18615 raft_consensus.cc:1270] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer 88ef62b247304e079788e24ddfdaa4fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:38.656975 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb" is_pre_election: true
I20250112 20:58:38.657615 18541 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:38.658107 19167 raft_consensus.cc:2798] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.658458 19167 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.658785 19167 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.661139 18436 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:38.661881 19017 raft_consensus.cc:2798] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.662288 19017 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.662711 19017 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.662648 18615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
W20250112 20:58:38.663322 18583 leader_election.cc:336] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.663393 18615 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:38.663923 18583 leader_election.cc:336] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
W20250112 20:58:38.667399 18509 leader_election.cc:336] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
W20250112 20:58:38.668094 18509 leader_election.cc:336] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.670738 19085 consensus_queue.cc:1035] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:38.676694 18615 raft_consensus.cc:2463] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 2.
I20250112 20:58:38.653216 19160 raft_consensus.cc:3054] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.681536 18509 leader_election.cc:304] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
W20250112 20:58:38.682366 18435 leader_election.cc:336] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.684800 18470 raft_consensus.cc:2463] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:38.683779 19136 raft_consensus.cc:2798] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election won for term 2
W20250112 20:58:38.691181 18435 leader_election.cc:336] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.691339 19136 raft_consensus.cc:695] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
W20250112 20:58:38.694098 18509 consensus_peers.cc:487] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:38.693936 19136 consensus_queue.cc:237] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.697326 19167 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.689608 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:38.689399 19017 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:38.699874 18471 raft_consensus.cc:2463] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 1.
I20250112 20:58:38.699760 19160 raft_consensus.cc:513] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.698444 18613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1332a302b6a44f9c8963dfed69c078a5" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:38.700973 18613 raft_consensus.cc:3054] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.702292 19017 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:38.703438 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "013cadee88a64ffe9b1b31198139d6d1" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:38.704041 18614 raft_consensus.cc:3054] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.705946 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
W20250112 20:58:38.706190 18435 leader_election.cc:336] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.709390 18613 raft_consensus.cc:2463] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 2.
I20250112 20:58:38.710631 18541 raft_consensus.cc:2388] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2f9ce9a0415c4546b9a764ac06cefb38 in current term 2: Already voted for candidate 88ef62b247304e079788e24ddfdaa4fb in this term.
W20250112 20:58:38.711194 18509 leader_election.cc:336] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.711400 19167 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.712570 18471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:38.713332 18471 raft_consensus.cc:2388] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 88ef62b247304e079788e24ddfdaa4fb in current term 2: Already voted for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in this term.
I20250112 20:58:38.714480 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59cd634f5244017b18a111892243f57" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c" is_pre_election: true
I20250112 20:58:38.715207 18618 raft_consensus.cc:2463] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 1.
I20250112 20:58:38.714593 18436 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38; no voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3
W20250112 20:58:38.715907 18509 leader_election.cc:336] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.716316 18509 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.717721 18435 leader_election.cc:304] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.717875 19165 raft_consensus.cc:2743] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.718161 18585 leader_election.cc:304] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.718755 19167 raft_consensus.cc:2743] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250112 20:58:38.718978 18718 raft_consensus.cc:2798] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.718930 19160 leader_election.cc:290] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.720795 19179 raft_consensus.cc:2798] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.721282 19179 raft_consensus.cc:695] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:38.722679 18471 raft_consensus.cc:1270] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer 88ef62b247304e079788e24ddfdaa4fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:38.722332 19179 consensus_queue.cc:237] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.724486 19160 consensus_queue.cc:1035] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:38.724937 18614 raft_consensus.cc:2463] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 2.
I20250112 20:58:38.725912 18509 leader_election.cc:304] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.726944 19160 raft_consensus.cc:2798] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.727543 19160 raft_consensus.cc:695] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:38.727504 18585 leader_election.cc:304] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.728662 19160 consensus_queue.cc:237] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:38.730283 19180 raft_consensus.cc:2798] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.730682 19180 raft_consensus.cc:491] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.731010 19180 raft_consensus.cc:3054] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.738907 19180 raft_consensus.cc:513] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.745975 19180 leader_election.cc:290] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:38.748113 18470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b92c85eeea5465ea50fa57eb9214098" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:38.748834 18470 raft_consensus.cc:3054] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.756420 18470 raft_consensus.cc:2463] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:38.757505 18585 leader_election.cc:304] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:38.758358 19180 raft_consensus.cc:2798] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:38.758816 19180 raft_consensus.cc:695] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:38.759670 19180 consensus_queue.cc:237] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:38.782423 18583 leader_election.cc:336] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:38.812924 18290 catalog_manager.cc:5526] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 2 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:38.821125 18291 catalog_manager.cc:5526] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:38.822925 18291 catalog_manager.cc:5526] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:38.861244 18718 raft_consensus.cc:695] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
W20250112 20:58:38.861972 19184 log.cc:926] Time spent T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb: Append to log took a long time: real 0.072s user 0.001s sys 0.000s
I20250112 20:58:38.862318 18718 consensus_queue.cc:237] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
W20250112 20:58:38.867025 19183 log.cc:926] Time spent T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c: Append to log took a long time: real 0.079s user 0.001s sys 0.000s
W20250112 20:58:38.877882 18509 consensus_peers.cc:487] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:38.896988 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 177490us
I20250112 20:58:38.903164 18435 leader_election.cc:304] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:38.903975 19017 raft_consensus.cc:2798] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250112 20:58:38.904407 19017 raft_consensus.cc:491] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:58:38.904781 19017 raft_consensus.cc:3054] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:38.910288 18615 raft_consensus.cc:1270] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer 2f9ce9a0415c4546b9a764ac06cefb38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:38.915448 19017 raft_consensus.cc:513] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:38.948555 19017 leader_election.cc:290] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:38.948509 18290 catalog_manager.cc:5526] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 2 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:38.955036 18471 raft_consensus.cc:1270] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:38.961846 19179 consensus_queue.cc:1035] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20250112 20:58:38.964151 18583 consensus_peers.cc:487] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:38.964711 18509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 62055.7us
I20250112 20:58:38.968924 18291 catalog_manager.cc:5526] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:38.982090 18718 consensus_queue.cc:1035] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:38.982721 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f59cd634f5244017b18a111892243f57" candidate_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:38.983628 18614 raft_consensus.cc:3054] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:39.000268 18509 consensus_peers.cc:487] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:39.003110 18541 raft_consensus.cc:1270] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.005345 19180 consensus_queue.cc:1035] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:39.007179 18468 raft_consensus.cc:1270] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer 88ef62b247304e079788e24ddfdaa4fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.012202 19193 consensus_queue.cc:1035] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:39.013878 18614 raft_consensus.cc:2463] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9ce9a0415c4546b9a764ac06cefb38 in term 2.
W20250112 20:58:39.029552 18583 consensus_peers.cc:487] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:39.062031 18468 raft_consensus.cc:1270] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer 88ef62b247304e079788e24ddfdaa4fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.064307 19199 consensus_queue.cc:1035] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20250112 20:58:39.067209 18509 consensus_peers.cc:487] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:39.069651 18435 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.132:40357, user_credentials={real_user=slave}} blocked reactor thread for 88568.9us
I20250112 20:58:39.072878 18435 leader_election.cc:304] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:39.073616 18718 raft_consensus.cc:2798] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:39.074056 18718 raft_consensus.cc:695] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Becoming Leader. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:39.074755 18718 consensus_queue.cc:237] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:39.082697 18614 raft_consensus.cc:1270] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer 2f9ce9a0415c4546b9a764ac06cefb38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20250112 20:58:39.085496 18435 leader_election.cc:336] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
W20250112 20:58:39.086259 18435 consensus_peers.cc:487] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:39.086690 18435 consensus_peers.cc:487] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:39.089514 19186 consensus_queue.cc:1035] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:39.108522 18290 catalog_manager.cc:5526] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 reported cstate change: term changed from 1 to 2, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: HEALTHY } } }
W20250112 20:58:39.119251 18509 consensus_peers.cc:487] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:39.171665 18470 raft_consensus.cc:1270] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.172878 19180 consensus_queue.cc:1035] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:39.185810 18583 consensus_peers.cc:487] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:39.275555 18614 raft_consensus.cc:1270] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer 2f9ce9a0415c4546b9a764ac06cefb38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.276791 18718 consensus_queue.cc:1035] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:39.279901 18583 consensus_peers.cc:487] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:39.287322 15454 thread.cc:535] Waited for 2000ms trying to join with diag-logger (tid 18417)
I20250112 20:58:39.294462 18614 raft_consensus.cc:1270] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer 88ef62b247304e079788e24ddfdaa4fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.296003 19136 consensus_queue.cc:1035] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:39.297940 18435 consensus_peers.cc:487] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:39.311081 18613 raft_consensus.cc:1270] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer 88ef62b247304e079788e24ddfdaa4fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.312749 19136 consensus_queue.cc:1035] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:39.323057 18509 consensus_peers.cc:487] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:39.340005 18509 consensus_peers.cc:487] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250112 20:58:39.368299 18468 raft_consensus.cc:1270] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.369745 18995 consensus_queue.cc:1035] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:39.440122 18509 consensus_peers.cc:487] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.454668 18435 consensus_peers.cc:487] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.456301 18435 consensus_peers.cc:487] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.457819 18509 consensus_peers.cc:487] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.461850 18435 consensus_peers.cc:487] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.513904 18509 consensus_peers.cc:487] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.552942 18435 consensus_peers.cc:487] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.555632 18509 consensus_peers.cc:487] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.560880 18583 consensus_peers.cc:487] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.562284 18435 consensus_peers.cc:487] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:39.573280 18435 consensus_peers.cc:487] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.575313 18435 consensus_peers.cc:487] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.593379 18435 consensus_peers.cc:487] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.606932 18583 consensus_peers.cc:487] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.610453 18435 consensus_peers.cc:487] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250112 20:58:39.695998 18614 raft_consensus.cc:1270] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer 2f9ce9a0415c4546b9a764ac06cefb38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:39.696972 19017 consensus_queue.cc:1035] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:39.708765 18435 consensus_peers.cc:487] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.736820 18583 consensus_peers.cc:487] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.781167 18509 consensus_peers.cc:487] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.792318 18435 consensus_peers.cc:487] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.823621 18435 consensus_peers.cc:487] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.859963 18509 consensus_peers.cc:487] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.875725 18583 consensus_peers.cc:487] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.925451 18509 consensus_peers.cc:487] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.954447 18435 consensus_peers.cc:487] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:39.955996 18509 consensus_peers.cc:487] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:40.013545 18509 consensus_peers.cc:487] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:40.101820 18435 consensus_peers.cc:487] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:40.113359 18435 consensus_peers.cc:487] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:40.205282 18435 consensus_peers.cc:487] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:40.234772 18435 consensus_peers.cc:487] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:40.287786 15454 thread.cc:535] Waited for 3000ms trying to join with diag-logger (tid 18417)
I20250112 20:58:40.500353 19160 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:58:40.500922 19160 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:40.503428 19160 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:40.504352 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" is_pre_election: true
I20250112 20:58:40.505028 18468 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 2.
I20250112 20:58:40.506121 18511 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters:
I20250112 20:58:40.507014 19193 raft_consensus.cc:2798] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader pre-election won for term 3
W20250112 20:58:40.507261 18509 leader_election.cc:336] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:40.507385 19193 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:58:40.507808 19193 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:40.517879 19193 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:40.519665 19193 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 3 election: Requested vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:40.520576 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "88ef62b247304e079788e24ddfdaa4fb" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:40.521214 18468 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Advancing to term 3
W20250112 20:58:40.523665 18509 leader_election.cc:336] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:58:40.527449 18468 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88ef62b247304e079788e24ddfdaa4fb in term 3.
I20250112 20:58:40.529034 18511 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, 88ef62b247304e079788e24ddfdaa4fb; no voters: a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:40.529784 19215 raft_consensus.cc:2798] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:40.530153 19215 raft_consensus.cc:695] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 3 LEADER]: Becoming Leader. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:40.530822 19215 consensus_queue.cc:237] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:40.541617 18291 catalog_manager.cc:5526] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb reported cstate change: term changed from 1 to 3, leader changed from a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129) to 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131). New cstate: current_term: 3 leader_uuid: "88ef62b247304e079788e24ddfdaa4fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:41.010777 18468 raft_consensus.cc:1270] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 3 FOLLOWER]: Refusing update from remote peer 88ef62b247304e079788e24ddfdaa4fb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250112 20:58:41.012228 19193 consensus_queue.cc:1035] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250112 20:58:41.034623 18509 consensus_peers.cc:487] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:41.119841 18509 consensus_peers.cc:487] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.154953 18509 consensus_peers.cc:487] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.288293 15454 thread.cc:535] Waited for 4000ms trying to join with diag-logger (tid 18417)
W20250112 20:58:41.423389 18583 consensus_peers.cc:487] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.532158 18435 consensus_peers.cc:487] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.593699 18583 consensus_peers.cc:487] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.597060 18583 consensus_peers.cc:487] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.632056 18509 consensus_peers.cc:487] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.633494 18509 consensus_peers.cc:487] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:41.709219 18583 consensus_peers.cc:487] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.749625 18509 consensus_peers.cc:487] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.756088 18509 consensus_peers.cc:487] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.757570 18435 consensus_peers.cc:487] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.819029 18509 consensus_peers.cc:487] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.862380 18435 consensus_peers.cc:487] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:41.928189 18583 consensus_peers.cc:487] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:41.932384 18509 consensus_peers.cc:487] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:41.955314 18509 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111) [suppressed 428 similar messages]
W20250112 20:58:41.972041 18435 consensus_peers.cc:487] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:41.983958 18435 consensus_peers.cc:487] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.033288 18435 consensus_peers.cc:487] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.040895 18435 consensus_peers.cc:487] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250112 20:58:42.044669 18435 consensus_peers.cc:487] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.046705 18435 consensus_peers.cc:487] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.125209 18435 consensus_peers.cc:487] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.144764 18509 consensus_peers.cc:487] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.174043 18583 consensus_peers.cc:487] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.177646 18435 consensus_peers.cc:487] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.180107 18435 consensus_peers.cc:487] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.184912 18435 consensus_peers.cc:487] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.191532 18509 consensus_peers.cc:487] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.215480 18435 consensus_peers.cc:487] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.288784 15454 thread.cc:535] Waited for 5000ms trying to join with diag-logger (tid 18417)
W20250112 20:58:42.302984 18509 consensus_peers.cc:487] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.304931 18509 consensus_peers.cc:487] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.308281 18509 consensus_peers.cc:487] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.389357 18435 consensus_peers.cc:487] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.440686 18435 consensus_peers.cc:487] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.442234 18435 consensus_peers.cc:487] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.554268 18509 consensus_peers.cc:487] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.568197 18509 consensus_peers.cc:487] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.617187 18583 consensus_peers.cc:487] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.627990 18583 consensus_peers.cc:487] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.738240 18435 consensus_peers.cc:487] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.750819 18435 consensus_peers.cc:487] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:42.773325 18509 consensus_peers.cc:487] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:43.058221 18435 consensus_peers.cc:487] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:43.289212 15454 thread.cc:535] Waited for 6000ms trying to join with diag-logger (tid 18417)
I20250112 20:58:43.643301 15454 tablet_server.cc:195] TabletServer@127.15.23.129:0 shutdown complete.
I20250112 20:58:43.706215 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:58:43.714030 19232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:58:43.714972 19233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:58:43.716074 19235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:58:43.716708 15454 server_base.cc:1034] running on GCE node
I20250112 20:58:43.718612 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:58:43.718891 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:58:43.719084 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715523719067 us; error 0 us; skew 500 ppm
I20250112 20:58:43.719748 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:58:43.727725 15454 webserver.cc:458] Webserver started at http://127.15.23.129:37837/ using document root <none> and password file <none>
I20250112 20:58:43.728336 15454 fs_manager.cc:362] Metadata directory not provided
W20250112 20:58:43.728504 18509 consensus_peers.cc:487] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250112 20:58:43.728626 15454 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250112 20:58:43.732956 15454 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20250112 20:58:43.740226 19240 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:58:43.740973 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250112 20:58:43.741300 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
format_stamp: "Formatted at 2025-01-12 20:58:25 on dist-test-slave-m2nm"
I20250112 20:58:43.741740 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1736715392595610-15454-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:58:43.764326 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:58:43.765573 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:58:43.766983 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:58:43.844893 19249 ts_tablet_manager.cc:542] Loading tablet metadata (0/45 complete)
W20250112 20:58:43.881973 18509 consensus_peers.cc:487] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:43.898720 18509 consensus_peers.cc:487] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:43.942763 18583 consensus_peers.cc:487] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:43.975379 18583 consensus_peers.cc:487] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:43.983096 18583 consensus_peers.cc:487] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:44.076730 18435 consensus_peers.cc:487] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250112 20:58:44.094252 18583 consensus_peers.cc:487] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250112 20:58:44.124316 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20250112 20:58:44.124665 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.354s user 0.020s sys 0.019s
I20250112 20:58:44.125020 15454 ts_tablet_manager.cc:594] Registering tablets (0/45 complete)
I20250112 20:58:44.130975 19249 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.136931 18509 consensus_peers.cc:487] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.145927 19249 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.146641 19249 tablet_bootstrap.cc:492] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.147094 19249 ts_tablet_manager.cc:1397] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20250112 20:58:44.149366 19249 raft_consensus.cc:357] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.150264 19249 raft_consensus.cc:738] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.150784 19249 consensus_queue.cc:260] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.153194 19249 ts_tablet_manager.cc:1428] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20250112 20:58:44.153775 19249 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.167353 19249 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.168282 19249 tablet_bootstrap.cc:492] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.168915 19249 ts_tablet_manager.cc:1397] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20250112 20:58:44.171128 19249 raft_consensus.cc:357] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.171787 19249 raft_consensus.cc:738] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.172416 19249 consensus_queue.cc:260] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.174361 19249 ts_tablet_manager.cc:1428] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20250112 20:58:44.175021 19249 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.179426 18435 consensus_peers.cc:487] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20250112 20:58:44.182330 18509 consensus_peers.cc:487] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250112 20:58:44.188175 19249 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.189015 19249 tablet_bootstrap.cc:492] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.189476 19249 ts_tablet_manager.cc:1397] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.003s
I20250112 20:58:44.191465 19249 raft_consensus.cc:357] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.192088 19249 raft_consensus.cc:738] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.192616 19249 consensus_queue.cc:260] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.193989 19249 ts_tablet_manager.cc:1428] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20250112 20:58:44.194514 19249 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.205502 19249 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.206117 19249 tablet_bootstrap.cc:492] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.206574 19249 ts_tablet_manager.cc:1397] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20250112 20:58:44.208047 19249 raft_consensus.cc:357] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.208493 19249 raft_consensus.cc:738] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.209077 19249 consensus_queue.cc:260] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.210439 19249 ts_tablet_manager.cc:1428] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20250112 20:58:44.210979 19249 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.222743 19249 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.223453 19249 tablet_bootstrap.cc:492] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.224020 19249 ts_tablet_manager.cc:1397] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20250112 20:58:44.225786 19249 raft_consensus.cc:357] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.226341 19249 raft_consensus.cc:738] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.226939 19249 consensus_queue.cc:260] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.228928 19249 ts_tablet_manager.cc:1428] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20250112 20:58:44.229609 19249 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.243957 19249 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.244810 19249 tablet_bootstrap.cc:492] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.245409 19249 ts_tablet_manager.cc:1397] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20250112 20:58:44.247805 19249 raft_consensus.cc:357] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.248528 19249 raft_consensus.cc:738] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.249255 19249 consensus_queue.cc:260] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.251250 19249 ts_tablet_manager.cc:1428] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:58:44.251912 19249 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.269587 18509 consensus_peers.cc:487] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250112 20:58:44.290628 19249 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.291363 19249 tablet_bootstrap.cc:492] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.291831 19249 ts_tablet_manager.cc:1397] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.040s user 0.024s sys 0.000s
I20250112 20:58:44.293728 19249 raft_consensus.cc:357] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.294216 19249 raft_consensus.cc:738] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.294780 19249 consensus_queue.cc:260] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.296129 19249 ts_tablet_manager.cc:1428] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250112 20:58:44.296695 19249 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.312237 19249 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.314718 19249 tablet_bootstrap.cc:492] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.315295 19249 ts_tablet_manager.cc:1397] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.003s
W20250112 20:58:44.315594 18435 consensus_peers.cc:487] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20250112 20:58:44.316082 18435 consensus_peers.cc:487] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250112 20:58:44.317222 19249 raft_consensus.cc:357] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.317750 19249 raft_consensus.cc:738] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.318207 19249 consensus_queue.cc:260] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.319588 19249 ts_tablet_manager.cc:1428] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250112 20:58:44.320107 19249 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.331231 18509 consensus_peers.cc:487] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250112 20:58:44.334453 15454 ts_tablet_manager.cc:610] Registered 45 tablets
I20250112 20:58:44.334777 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.210s user 0.195s sys 0.008s
I20250112 20:58:44.337608 19249 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.338445 19249 tablet_bootstrap.cc:492] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.339064 19249 ts_tablet_manager.cc:1397] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.004s
I20250112 20:58:44.341236 19249 raft_consensus.cc:357] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.341719 19249 raft_consensus.cc:738] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.342233 19249 consensus_queue.cc:260] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.343763 19249 ts_tablet_manager.cc:1428] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250112 20:58:44.344355 19249 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.358007 19249 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.359336 19249 tablet_bootstrap.cc:492] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.360261 19249 ts_tablet_manager.cc:1397] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20250112 20:58:44.363356 19249 raft_consensus.cc:357] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.364168 19249 raft_consensus.cc:738] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.364912 19249 consensus_queue.cc:260] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.367389 19249 ts_tablet_manager.cc:1428] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250112 20:58:44.368240 19249 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.415935 18435 consensus_peers.cc:487] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20250112 20:58:44.416056 19249 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.417014 19249 tablet_bootstrap.cc:492] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.417665 19249 ts_tablet_manager.cc:1397] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.050s user 0.020s sys 0.019s
I20250112 20:58:44.420645 19249 raft_consensus.cc:357] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.421336 19249 raft_consensus.cc:738] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.422037 19249 consensus_queue.cc:260] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.424299 19249 ts_tablet_manager.cc:1428] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:58:44.425042 19249 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.438634 19249 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.439714 19249 tablet_bootstrap.cc:492] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.440351 19249 ts_tablet_manager.cc:1397] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20250112 20:58:44.442643 19249 raft_consensus.cc:357] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.443495 19249 raft_consensus.cc:738] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.444154 19249 consensus_queue.cc:260] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.445734 19249 ts_tablet_manager.cc:1428] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250112 20:58:44.446453 19249 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.457943 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.129:38493
I20250112 20:58:44.458213 19320 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.129:38493 every 8 connection(s)
I20250112 20:58:44.461553 19249 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.463526 19249 tablet_bootstrap.cc:492] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.464190 19249 ts_tablet_manager.cc:1397] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20250112 20:58:44.466750 19249 raft_consensus.cc:357] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.467440 19249 raft_consensus.cc:738] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.468052 19249 consensus_queue.cc:260] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.469785 19249 ts_tablet_manager.cc:1428] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250112 20:58:44.470546 19249 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.502034 18435 consensus_peers.cc:487] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.507452 19249 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20250112 20:58:44.507908 18509 consensus_peers.cc:487] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20250112 20:58:44.508586 19249 tablet_bootstrap.cc:492] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.509420 19249 ts_tablet_manager.cc:1397] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.039s user 0.020s sys 0.015s
I20250112 20:58:44.512001 19249 raft_consensus.cc:357] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.512765 19249 raft_consensus.cc:738] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.513432 19249 consensus_queue.cc:260] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.515708 19249 ts_tablet_manager.cc:1428] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:58:44.516475 19249 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.517582 18435 consensus_peers.cc:487] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20250112 20:58:44.524166 18583 consensus_peers.cc:487] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.527083 19321 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45335
I20250112 20:58:44.527473 19321 heartbeater.cc:463] Registering TS with master...
I20250112 20:58:44.528187 19321 heartbeater.cc:510] Master 127.15.23.190:45335 requested a full tablet report, sending...
W20250112 20:58:44.529465 18435 consensus_peers.cc:487] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20250112 20:58:44.530177 18435 consensus_peers.cc:487] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.534678 19249 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.535552 19249 tablet_bootstrap.cc:492] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.536144 19249 ts_tablet_manager.cc:1397] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.004s
I20250112 20:58:44.538573 19249 raft_consensus.cc:357] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.539263 19249 raft_consensus.cc:738] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.539883 19249 consensus_queue.cc:260] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.541782 19249 ts_tablet_manager.cc:1428] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.005s sys 0.003s
I20250112 20:58:44.542394 19249 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.545619 18290 ts_manager.cc:194] Re-registered known tserver with Master: a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493)
I20250112 20:58:44.556268 19249 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.556797 19293 raft_consensus.cc:3054] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:44.557024 19249 tablet_bootstrap.cc:492] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.557649 19249 ts_tablet_manager.cc:1397] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20250112 20:58:44.559615 19249 raft_consensus.cc:357] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.560062 19249 raft_consensus.cc:738] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.561389 19292 raft_consensus.cc:3054] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:44.561149 19249 consensus_queue.cc:260] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.563446 19249 ts_tablet_manager.cc:1428] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20250112 20:58:44.564190 19249 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.578202 18583 consensus_peers.cc:487] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.594347 19249 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.595307 19249 tablet_bootstrap.cc:492] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.595932 19249 ts_tablet_manager.cc:1397] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.032s user 0.017s sys 0.005s
I20250112 20:58:44.598263 19249 raft_consensus.cc:357] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.599002 19249 raft_consensus.cc:738] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.599634 19249 consensus_queue.cc:260] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.601745 19249 ts_tablet_manager.cc:1428] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250112 20:58:44.601701 18290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54986
I20250112 20:58:44.602684 19249 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.606093 19321 heartbeater.cc:502] Master 127.15.23.190:45335 was elected leader, sending a full tablet report...
I20250112 20:58:44.622433 19249 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.623152 19249 tablet_bootstrap.cc:492] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.623682 19249 ts_tablet_manager.cc:1397] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.021s user 0.012s sys 0.000s
I20250112 20:58:44.625331 19249 raft_consensus.cc:357] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.625809 19249 raft_consensus.cc:738] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.627013 19249 consensus_queue.cc:260] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.628832 19249 ts_tablet_manager.cc:1428] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250112 20:58:44.629343 19249 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.640091 19249 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.640869 19249 tablet_bootstrap.cc:492] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.641408 19249 ts_tablet_manager.cc:1397] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
W20250112 20:58:44.641713 18509 consensus_peers.cc:487] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.643694 19249 raft_consensus.cc:357] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.644449 19249 raft_consensus.cc:738] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.645117 19249 consensus_queue.cc:260] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.647130 19249 ts_tablet_manager.cc:1428] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250112 20:58:44.647815 19249 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.661688 19249 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.662535 19249 tablet_bootstrap.cc:492] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.663187 19249 ts_tablet_manager.cc:1397] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20250112 20:58:44.665457 19249 raft_consensus.cc:357] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.665995 19249 raft_consensus.cc:738] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.666572 19249 consensus_queue.cc:260] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.668633 19249 ts_tablet_manager.cc:1428] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20250112 20:58:44.669315 19249 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.681926 19249 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.682678 19249 tablet_bootstrap.cc:492] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.683326 19249 ts_tablet_manager.cc:1397] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20250112 20:58:44.685670 19249 raft_consensus.cc:357] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.686610 19249 raft_consensus.cc:738] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.687252 19249 consensus_queue.cc:260] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.689605 19249 ts_tablet_manager.cc:1428] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:58:44.690300 19249 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.693507 18435 consensus_peers.cc:487] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.703374 19249 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.704268 19249 tablet_bootstrap.cc:492] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.704879 19249 ts_tablet_manager.cc:1397] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20250112 20:58:44.706935 19249 raft_consensus.cc:357] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.707405 19249 raft_consensus.cc:738] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.707849 19249 consensus_queue.cc:260] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.709782 19249 ts_tablet_manager.cc:1428] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250112 20:58:44.710863 19249 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.719070 19292 raft_consensus.cc:3054] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:44.724434 19249 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.725361 19249 tablet_bootstrap.cc:492] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.725950 19249 ts_tablet_manager.cc:1397] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20250112 20:58:44.728542 19293 raft_consensus.cc:3054] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:44.728840 19249 raft_consensus.cc:357] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.729614 19249 raft_consensus.cc:738] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.730213 19249 consensus_queue.cc:260] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.732262 19249 ts_tablet_manager.cc:1428] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20250112 20:58:44.733872 19249 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.748955 19249 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.750048 19249 tablet_bootstrap.cc:492] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.750631 19249 ts_tablet_manager.cc:1397] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20250112 20:58:44.752837 19249 raft_consensus.cc:357] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.753634 19249 raft_consensus.cc:738] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.754303 19249 consensus_queue.cc:260] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.755820 19249 ts_tablet_manager.cc:1428] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20250112 20:58:44.756322 19249 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.768247 19249 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.768975 19249 tablet_bootstrap.cc:492] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.769436 19249 ts_tablet_manager.cc:1397] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.012s
I20250112 20:58:44.770954 19249 raft_consensus.cc:357] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.771399 19249 raft_consensus.cc:738] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.771876 19249 consensus_queue.cc:260] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.773785 19249 ts_tablet_manager.cc:1428] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20250112 20:58:44.774952 19249 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.788009 19249 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.788923 19249 tablet_bootstrap.cc:492] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.788872 19293 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:44.789845 19249 ts_tablet_manager.cc:1397] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20250112 20:58:44.793155 19249 raft_consensus.cc:357] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.793820 19249 raft_consensus.cc:738] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.794426 19249 consensus_queue.cc:260] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.796241 19249 ts_tablet_manager.cc:1428] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20250112 20:58:44.796839 19249 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.802443 19293 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 3
I20250112 20:58:44.810686 19249 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.811326 19249 tablet_bootstrap.cc:492] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.811769 19249 ts_tablet_manager.cc:1397] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20250112 20:58:44.813874 19249 raft_consensus.cc:357] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.814447 19249 raft_consensus.cc:738] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.814917 19249 consensus_queue.cc:260] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.816725 19249 ts_tablet_manager.cc:1428] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20250112 20:58:44.817332 19249 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.828362 19249 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.829010 19249 tablet_bootstrap.cc:492] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.829442 19249 ts_tablet_manager.cc:1397] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20250112 20:58:44.831867 19249 raft_consensus.cc:357] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.832554 19249 raft_consensus.cc:738] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.833182 19249 consensus_queue.cc:260] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.834830 19249 ts_tablet_manager.cc:1428] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20250112 20:58:44.835536 19249 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.849386 19249 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.849646 19293 raft_consensus.cc:3054] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:44.850111 19249 tablet_bootstrap.cc:492] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.850703 19249 ts_tablet_manager.cc:1397] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.002s
I20250112 20:58:44.850986 19292 raft_consensus.cc:3054] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:44.853084 19249 raft_consensus.cc:357] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.853845 19249 raft_consensus.cc:738] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.854490 19249 consensus_queue.cc:260] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.856820 19249 ts_tablet_manager.cc:1428] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20250112 20:58:44.857627 19249 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.871171 19249 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.871842 19249 tablet_bootstrap.cc:492] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.872325 19249 ts_tablet_manager.cc:1397] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20250112 20:58:44.874209 19249 raft_consensus.cc:357] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.874866 19249 raft_consensus.cc:738] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.875464 19249 consensus_queue.cc:260] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.877350 19249 ts_tablet_manager.cc:1428] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.004s sys 0.004s
I20250112 20:58:44.878113 19249 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.903697 19249 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.904361 19249 tablet_bootstrap.cc:492] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.904848 19249 ts_tablet_manager.cc:1397] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.027s user 0.013s sys 0.012s
I20250112 20:58:44.906473 19249 raft_consensus.cc:357] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.906903 19249 raft_consensus.cc:738] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.907298 19249 consensus_queue.cc:260] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:44.908871 19249 ts_tablet_manager.cc:1428] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250112 20:58:44.909457 19249 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.924474 19249 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.926220 19249 tablet_bootstrap.cc:492] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.926868 19249 ts_tablet_manager.cc:1397] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.000s
I20250112 20:58:44.929016 19249 raft_consensus.cc:357] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.929656 19249 raft_consensus.cc:738] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.930243 19249 consensus_queue.cc:260] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:44.931820 19249 ts_tablet_manager.cc:1428] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20250112 20:58:44.932355 19249 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:44.946777 19249 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.947644 19249 tablet_bootstrap.cc:492] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.948143 19249 ts_tablet_manager.cc:1397] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20250112 20:58:44.949963 19249 raft_consensus.cc:357] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.950711 19249 raft_consensus.cc:738] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.951380 19249 consensus_queue.cc:260] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.954216 19249 ts_tablet_manager.cc:1428] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20250112 20:58:44.954916 19249 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:44.976344 18509 consensus_peers.cc:487] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.989060 19249 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:44.990216 19249 tablet_bootstrap.cc:492] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:44.990800 19249 ts_tablet_manager.cc:1397] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.036s user 0.017s sys 0.016s
W20250112 20:58:44.992105 18435 consensus_peers.cc:487] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:44.993017 19249 raft_consensus.cc:357] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.993573 19249 raft_consensus.cc:738] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:44.994156 19249 consensus_queue.cc:260] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:44.995604 19249 ts_tablet_manager.cc:1428] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250112 20:58:44.996115 19249 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.006940 19249 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20250112 20:58:45.007043 18435 consensus_peers.cc:487] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:45.008105 19249 tablet_bootstrap.cc:492] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.008817 19249 ts_tablet_manager.cc:1397] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.003s
I20250112 20:58:45.011489 19292 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:45.012092 19249 raft_consensus.cc:357] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:45.012818 19249 raft_consensus.cc:738] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.013439 19249 consensus_queue.cc:260] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:45.015066 19249 ts_tablet_manager.cc:1428] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:58:45.015774 19249 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
W20250112 20:58:45.025393 18435 consensus_peers.cc:487] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250112 20:58:45.037056 19249 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.037772 19249 tablet_bootstrap.cc:492] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.038242 19249 ts_tablet_manager.cc:1397] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.008s
I20250112 20:58:45.040016 19249 raft_consensus.cc:357] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:45.040467 19249 raft_consensus.cc:738] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.041026 19249 consensus_queue.cc:260] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:45.042565 19249 ts_tablet_manager.cc:1428] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250112 20:58:45.043784 19249 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.060415 19249 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.061316 19249 tablet_bootstrap.cc:492] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.061887 19249 ts_tablet_manager.cc:1397] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.007s
I20250112 20:58:45.063889 19249 raft_consensus.cc:357] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:45.064375 19249 raft_consensus.cc:738] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.064927 19249 consensus_queue.cc:260] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:45.067132 19249 ts_tablet_manager.cc:1428] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20250112 20:58:45.067822 19249 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.079394 19249 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.080354 19249 tablet_bootstrap.cc:492] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.081041 19249 ts_tablet_manager.cc:1397] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20250112 20:58:45.080967 19293 raft_consensus.cc:3054] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:45.083563 19249 raft_consensus.cc:357] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:45.084326 19249 raft_consensus.cc:738] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.084959 19249 consensus_queue.cc:260] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:45.086938 19249 ts_tablet_manager.cc:1428] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20250112 20:58:45.087635 19249 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.101279 19249 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.102099 19249 tablet_bootstrap.cc:492] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.102768 19249 ts_tablet_manager.cc:1397] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20250112 20:58:45.105015 19249 raft_consensus.cc:357] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:45.105623 19249 raft_consensus.cc:738] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.106168 19249 consensus_queue.cc:260] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:45.107751 19249 ts_tablet_manager.cc:1428] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20250112 20:58:45.108821 19249 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.125329 19249 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.125962 19249 tablet_bootstrap.cc:492] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.126451 19249 ts_tablet_manager.cc:1397] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.000s
I20250112 20:58:45.128957 19249 raft_consensus.cc:357] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:45.129659 19249 raft_consensus.cc:738] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.130259 19249 consensus_queue.cc:260] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:45.131968 19249 ts_tablet_manager.cc:1428] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250112 20:58:45.132568 19249 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.142669 19293 raft_consensus.cc:3054] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:45.145817 19249 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.146759 19249 tablet_bootstrap.cc:492] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.147342 19249 ts_tablet_manager.cc:1397] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20250112 20:58:45.149544 19249 raft_consensus.cc:357] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:45.150245 19249 raft_consensus.cc:738] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.150781 19249 consensus_queue.cc:260] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:45.152817 19249 ts_tablet_manager.cc:1428] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250112 20:58:45.153432 19249 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.170435 19293 raft_consensus.cc:3054] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:45.172102 19249 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.173069 19249 tablet_bootstrap.cc:492] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.173710 19249 ts_tablet_manager.cc:1397] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.008s
I20250112 20:58:45.175894 19249 raft_consensus.cc:357] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:45.176532 19249 raft_consensus.cc:738] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.177069 19249 consensus_queue.cc:260] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:45.178804 19249 ts_tablet_manager.cc:1428] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20250112 20:58:45.179534 19249 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.192096 19249 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.192895 19249 tablet_bootstrap.cc:492] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.193497 19249 ts_tablet_manager.cc:1397] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.003s
I20250112 20:58:45.195631 19249 raft_consensus.cc:357] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:45.196228 19249 raft_consensus.cc:738] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.196811 19249 consensus_queue.cc:260] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:45.198945 19249 ts_tablet_manager.cc:1428] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20250112 20:58:45.200322 19249 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.212741 19249 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.213471 19249 tablet_bootstrap.cc:492] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.213975 19249 ts_tablet_manager.cc:1397] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20250112 20:58:45.216109 19249 raft_consensus.cc:357] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:45.216687 19249 raft_consensus.cc:738] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.217255 19249 consensus_queue.cc:260] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:45.219244 19249 ts_tablet_manager.cc:1428] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20250112 20:58:45.219935 19249 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap starting.
I20250112 20:58:45.231957 19249 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250112 20:58:45.232729 19249 tablet_bootstrap.cc:492] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Bootstrap complete.
I20250112 20:58:45.235056 19249 ts_tablet_manager.cc:1397] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20250112 20:58:45.237260 19249 raft_consensus.cc:357] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:45.237897 19249 raft_consensus.cc:738] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Initialized, Role: FOLLOWER
I20250112 20:58:45.238574 19249 consensus_queue.cc:260] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:45.240738 19249 ts_tablet_manager.cc:1428] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250112 20:58:45.336853 19293 raft_consensus.cc:3054] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:45.400681 19292 raft_consensus.cc:3054] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:45.453538 19292 raft_consensus.cc:3054] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
W20250112 20:58:46.538514 18436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.131:39861, user_credentials={real_user=slave}} blocked reactor thread for 48025.9us
W20250112 20:58:46.565189 18511 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:35469, user_credentials={real_user=slave}} blocked reactor thread for 134342us
W20250112 20:58:47.347265 18566 debug-util.cc:398] Leaking SignalData structure 0x7b08002adf60 after lost signal to thread 15457
W20250112 20:58:47.348428 18566 debug-util.cc:398] Leaking SignalData structure 0x7b08001354c0 after lost signal to thread 18328
W20250112 20:58:47.349596 18566 debug-util.cc:398] Leaking SignalData structure 0x7b08002260e0 after lost signal to thread 18495
W20250112 20:58:47.350596 18566 debug-util.cc:398] Leaking SignalData structure 0x7b0800318620 after lost signal to thread 18569
W20250112 20:58:47.351598 18566 debug-util.cc:398] Leaking SignalData structure 0x7b08002589e0 after lost signal to thread 18643
W20250112 20:58:47.352762 18566 debug-util.cc:398] Leaking SignalData structure 0x7b08002c1740 after lost signal to thread 19320
I20250112 20:58:54.617902 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:54.671399 18468 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:54.672070 18468 raft_consensus.cc:604] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:54.674436 18468 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:54.674873 18468 raft_consensus.cc:604] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:54.676750 18468 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6f3629a889ed427db7d163963dd13507"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:54.677135 18468 raft_consensus.cc:604] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:54.678954 18468 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:54.679414 18468 raft_consensus.cc:604] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:54.690255 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:54.690659 18545 raft_consensus.cc:604] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:54.692669 18468 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e8704b5088604084a641aa00baf84be8"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:54.693066 18468 raft_consensus.cc:604] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:54.693864 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20250112 20:58:54.694855 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:54.695317 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:54.744025 19417 raft_consensus.cc:988] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:54.744642 19418 raft_consensus.cc:1076] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:54.746133 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6f3629a889ed427db7d163963dd13507"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52604
I20250112 20:58:54.746611 19292 raft_consensus.cc:491] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:54.746933 19292 raft_consensus.cc:3054] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.752502 19292 raft_consensus.cc:513] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.754228 19292 leader_election.cc:290] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:54.761412 19418 raft_consensus.cc:988] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:54.762130 19416 raft_consensus.cc:1076] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:54.763366 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52604
I20250112 20:58:54.768944 19292 raft_consensus.cc:491] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:54.769325 19292 raft_consensus.cc:3054] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.769987 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3629a889ed427db7d163963dd13507" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:54.770728 18619 raft_consensus.cc:3054] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.770557 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f3629a889ed427db7d163963dd13507" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:54.771245 18468 raft_consensus.cc:3049] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:54.771574 18468 raft_consensus.cc:738] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:54.772526 18468 consensus_queue.cc:260] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.773890 18468 raft_consensus.cc:3054] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.777618 18619 raft_consensus.cc:2463] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:54.778890 19242 leader_election.cc:304] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:54.778488 19292 raft_consensus.cc:513] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.783155 19292 leader_election.cc:290] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:54.781271 18468 raft_consensus.cc:2463] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:54.785239 19425 raft_consensus.cc:2798] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:54.785900 18619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:54.786345 18619 raft_consensus.cc:3054] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.786600 19425 raft_consensus.cc:695] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:54.787815 19425 consensus_queue.cc:237] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.792526 18619 raft_consensus.cc:2463] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:54.793648 19242 leader_election.cc:304] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:54.794607 19426 raft_consensus.cc:2798] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:54.795254 19426 raft_consensus.cc:695] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:54.796612 19426 consensus_queue.cc:237] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.797343 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca1ca0f378247598c915c11faee8ed2" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:54.798242 18468 raft_consensus.cc:3049] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:54.798840 18468 raft_consensus.cc:738] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:54.799786 18468 consensus_queue.cc:260] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.801048 18468 raft_consensus.cc:3054] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.808863 18468 raft_consensus.cc:2463] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:54.820914 18293 catalog_manager.cc:5526] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:54.841691 18293 catalog_manager.cc:5526] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:54.845974 19402 raft_consensus.cc:988] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:54.846627 19400 raft_consensus.cc:1076] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:54.847812 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52612
I20250112 20:58:54.848215 19292 raft_consensus.cc:491] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:54.848560 19292 raft_consensus.cc:3054] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.854853 19292 raft_consensus.cc:513] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:54.857110 19292 leader_election.cc:290] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:54.858217 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:54.858851 18617 raft_consensus.cc:3054] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.863761 18617 raft_consensus.cc:2463] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:54.864514 19242 leader_election.cc:304] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:54.865252 19426 raft_consensus.cc:2798] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:54.865505 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7daf053c1144c6797f1ac89fedd0fd0" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:54.865799 19426 raft_consensus.cc:695] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:54.866096 18545 raft_consensus.cc:3049] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:54.866403 18545 raft_consensus.cc:738] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:54.866680 19426 consensus_queue.cc:237] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:54.867085 18545 consensus_queue.cc:260] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:54.868227 18545 raft_consensus.cc:3054] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.875404 18545 raft_consensus.cc:2463] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:54.879572 18293 catalog_manager.cc:5526] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:54.903115 19417 raft_consensus.cc:988] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:54.903537 19416 raft_consensus.cc:1076] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:54.905045 19287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52604
I20250112 20:58:54.905606 19287 raft_consensus.cc:491] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:54.905951 19287 raft_consensus.cc:3054] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.911368 19287 raft_consensus.cc:513] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.913157 19287 leader_election.cc:290] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:54.913307 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:54.913695 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb32432c5bc044b5b9f6871bbb30be64" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:54.913929 18617 raft_consensus.cc:3054] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.914247 18468 raft_consensus.cc:3049] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:54.914603 18468 raft_consensus.cc:738] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:54.915345 18468 consensus_queue.cc:260] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.916648 18468 raft_consensus.cc:3054] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:54.920022 18617 raft_consensus.cc:2463] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:54.921108 19242 leader_election.cc:304] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:54.922530 19426 raft_consensus.cc:2798] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:54.923697 19426 raft_consensus.cc:695] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:54.924120 18468 raft_consensus.cc:2463] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:54.924703 19426 consensus_queue.cc:237] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:54.934616 18293 catalog_manager.cc:5526] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:55.013476 19418 raft_consensus.cc:988] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:55.013963 19416 raft_consensus.cc:1076] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:55.015460 18617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e8704b5088604084a641aa00baf84be8"
dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:43574
I20250112 20:58:55.015978 18617 raft_consensus.cc:491] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:55.016346 18617 raft_consensus.cc:3054] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.022163 18617 raft_consensus.cc:513] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:55.024065 18617 leader_election.cc:290] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:55.024307 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:55.024514 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8704b5088604084a641aa00baf84be8" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:55.025068 18468 raft_consensus.cc:3049] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:55.025195 18545 raft_consensus.cc:3054] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.025460 18468 raft_consensus.cc:738] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:55.026157 18468 consensus_queue.cc:260] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:55.027310 18468 raft_consensus.cc:3054] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.031330 18545 raft_consensus.cc:2463] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:55.032487 18584 leader_election.cc:304] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:55.033371 19435 raft_consensus.cc:2798] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:55.034020 18468 raft_consensus.cc:2463] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:55.034081 19435 raft_consensus.cc:695] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:55.035159 19435 consensus_queue.cc:237] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:55.044965 18290 catalog_manager.cc:5526] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:55.055812 19418 raft_consensus.cc:988] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.056218 19416 raft_consensus.cc:1076] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.057312 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52604
I20250112 20:58:55.057694 19292 raft_consensus.cc:491] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:55.057922 19292 raft_consensus.cc:3054] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.063020 19292 raft_consensus.cc:513] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.064479 19292 leader_election.cc:290] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:55.065201 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:55.065464 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60856bd5ad724e3792e927130b6315cf" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:55.065824 18468 raft_consensus.cc:3049] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:55.065968 18545 raft_consensus.cc:3054] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.066169 18468 raft_consensus.cc:738] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:55.066941 18468 consensus_queue.cc:260] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.068033 18468 raft_consensus.cc:3054] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.071784 18545 raft_consensus.cc:2463] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:55.072801 19243 leader_election.cc:304] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:55.073410 19426 raft_consensus.cc:2798] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:55.073485 18468 raft_consensus.cc:2463] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:55.073807 19426 raft_consensus.cc:695] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:55.074647 19426 consensus_queue.cc:237] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.090206 18290 catalog_manager.cc:5526] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:55.194379 18617 raft_consensus.cc:1270] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.195334 19426 consensus_queue.cc:1035] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.204685 18468 raft_consensus.cc:1270] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.207916 19426 consensus_queue.cc:1035] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.291798 18617 raft_consensus.cc:1270] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.292922 19426 consensus_queue.cc:1035] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.300997 18466 raft_consensus.cc:1270] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.302649 19428 consensus_queue.cc:1035] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:55.343060 18466 raft_consensus.cc:1270] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.344628 19428 consensus_queue.cc:1035] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.349016 18614 raft_consensus.cc:1270] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.351032 19425 consensus_queue.cc:1035] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.371486 18614 raft_consensus.cc:1270] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.375313 19426 consensus_queue.cc:1035] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.379228 18541 raft_consensus.cc:1270] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.380491 19425 consensus_queue.cc:1035] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.436735 18466 raft_consensus.cc:1270] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.437826 19435 consensus_queue.cc:1035] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.454566 18545 raft_consensus.cc:1270] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.459158 19435 consensus_queue.cc:1035] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:58:55.490211 18466 raft_consensus.cc:1270] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.491370 19425 consensus_queue.cc:1035] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.501116 18545 raft_consensus.cc:1270] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:55.502278 19428 consensus_queue.cc:1035] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:55.726430 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:55.762048 18466 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:55.762502 18466 raft_consensus.cc:604] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:55.764822 18466 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:55.765209 18466 raft_consensus.cc:604] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:55.767187 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:55.767688 18545 raft_consensus.cc:604] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 3 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:55.769588 18466 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8300b885639841769a76417fb45f2db7"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:55.769935 18466 raft_consensus.cc:604] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:55.771481 18466 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "95e82471468e430b877403ce304eac78"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:55.771883 18466 raft_consensus.cc:604] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:55.773447 18466 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77"
dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:55066
I20250112 20:58:55.773782 18466 raft_consensus.cc:604] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:55.774755 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20250112 20:58:55.776189 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:55.776768 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:55.811976 19416 raft_consensus.cc:988] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.812407 19418 raft_consensus.cc:1076] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.813773 19287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52604
I20250112 20:58:55.814201 19287 raft_consensus.cc:491] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:55.814543 19287 raft_consensus.cc:3054] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.819931 19287 raft_consensus.cc:513] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:55.821880 19287 leader_election.cc:290] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:55.822103 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:55.822708 18617 raft_consensus.cc:3054] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.823064 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "523dfa2a1c9e4cdf9def87c077e7c7f1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:55.823514 18466 raft_consensus.cc:3049] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:55.823800 18466 raft_consensus.cc:738] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:55.824357 18466 consensus_queue.cc:260] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:55.825253 18466 raft_consensus.cc:3054] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.826927 18617 raft_consensus.cc:2463] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:55.827960 19242 leader_election.cc:304] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:55.828729 19428 raft_consensus.cc:2798] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:55.829142 19428 raft_consensus.cc:695] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:55.830219 19416 raft_consensus.cc:988] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:55.829952 19428 consensus_queue.cc:237] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:55.832218 18466 raft_consensus.cc:2463] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:55.833722 19418 raft_consensus.cc:1076] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:55.835111 18617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8300b885639841769a76417fb45f2db7"
dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:43574
I20250112 20:58:55.835637 18617 raft_consensus.cc:491] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:55.835942 18617 raft_consensus.cc:3054] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.841717 18617 raft_consensus.cc:513] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:55.843592 18617 leader_election.cc:290] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:55.844539 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8300b885639841769a76417fb45f2db7" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:55.845129 18466 raft_consensus.cc:3049] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:55.844422 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8300b885639841769a76417fb45f2db7" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:55.845438 18466 raft_consensus.cc:738] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:55.845656 18541 raft_consensus.cc:3054] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.846159 18466 consensus_queue.cc:260] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:55.846457 18290 catalog_manager.cc:5526] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:55.847383 18466 raft_consensus.cc:3054] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.852144 18541 raft_consensus.cc:2463] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:55.853258 18466 raft_consensus.cc:2463] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:55.853229 18584 leader_election.cc:304] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:55.854233 19453 raft_consensus.cc:2798] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:55.854907 19453 raft_consensus.cc:695] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:55.855937 19453 consensus_queue.cc:237] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:55.876070 19400 raft_consensus.cc:988] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.876526 19413 raft_consensus.cc:1076] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 3 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.877140 18290 catalog_manager.cc:5526] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:55.877643 19287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52612
I20250112 20:58:55.878134 19287 raft_consensus.cc:491] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:55.878427 19287 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:58:55.883430 19457 raft_consensus.cc:988] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.883873 19457 raft_consensus.cc:1076] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.884840 19287 raft_consensus.cc:513] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.885227 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52604
I20250112 20:58:55.885704 19292 raft_consensus.cc:491] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:55.886060 19292 raft_consensus.cc:3054] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.887605 18541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:55.888175 18541 raft_consensus.cc:3049] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 3 LEADER]: Stepping down as leader of term 3
I20250112 20:58:55.888428 18541 raft_consensus.cc:738] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:55.889097 18541 consensus_queue.cc:260] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.889936 18541 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:58:55.890393 19292 raft_consensus.cc:513] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.891867 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ccd5a01ca4d4a85a4b033d0dcba06" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:55.892478 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:55.892673 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08844b71d8c54556af7a5d625ed646c2" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:55.892983 18466 raft_consensus.cc:3054] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 3 FOLLOWER]: Advancing to term 4
I20250112 20:58:55.893076 18468 raft_consensus.cc:3049] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:55.893214 18545 raft_consensus.cc:3054] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.893497 18468 raft_consensus.cc:738] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:55.894268 18468 consensus_queue.cc:260] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.895084 18541 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 4.
I20250112 20:58:55.896061 19243 leader_election.cc:304] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:55.897070 19428 raft_consensus.cc:2798] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 4 FOLLOWER]: Leader election won for term 4
I20250112 20:58:55.897493 19428 raft_consensus.cc:695] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 4 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:55.898209 19428 consensus_queue.cc:237] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.899148 19287 leader_election.cc:290] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 4 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:55.900389 18545 raft_consensus.cc:2463] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:55.901312 19243 leader_election.cc:304] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, a45f68dd3d9043809f3bb06287f5d2d3; no voters:
I20250112 20:58:55.901667 18466 raft_consensus.cc:2463] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 4.
I20250112 20:58:55.902060 19426 raft_consensus.cc:2798] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:55.902495 19426 raft_consensus.cc:695] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:55.895314 18468 raft_consensus.cc:3054] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.903333 19426 consensus_queue.cc:237] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } }
I20250112 20:58:55.892102 19292 leader_election.cc:290] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:55.911381 18468 raft_consensus.cc:2463] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:55.915180 19416 raft_consensus.cc:988] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.915581 19418 raft_consensus.cc:1076] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:55.916822 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52604
I20250112 20:58:55.917186 19292 raft_consensus.cc:491] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:55.917454 19292 raft_consensus.cc:3054] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.917264 18290 catalog_manager.cc:5526] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 3 to 4, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 4 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:55.921360 19292 raft_consensus.cc:513] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:55.923483 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:55.923588 19292 leader_election.cc:290] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:55.923732 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45830b149244e2f8bec3cac5228eb77" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:55.924031 18614 raft_consensus.cc:3054] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.924335 18468 raft_consensus.cc:3049] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:55.924634 18468 raft_consensus.cc:738] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:55.925350 18468 consensus_queue.cc:260] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:55.927611 18468 raft_consensus.cc:3054] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:55.930029 18614 raft_consensus.cc:2463] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:55.930822 19242 leader_election.cc:304] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:55.931519 19428 raft_consensus.cc:2798] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:55.931931 19428 raft_consensus.cc:695] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:55.932693 19428 consensus_queue.cc:237] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:55.933611 18468 raft_consensus.cc:2463] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:55.934684 18290 catalog_manager.cc:5526] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:55.948076 18290 catalog_manager.cc:5526] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:56.154192 19418 raft_consensus.cc:988] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:56.154784 19457 raft_consensus.cc:1076] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:56.156445 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "95e82471468e430b877403ce304eac78"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52604
I20250112 20:58:56.156988 19292 raft_consensus.cc:491] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:56.157322 19292 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:56.161453 19292 raft_consensus.cc:513] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:56.163316 19292 leader_election.cc:290] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 3 election: Requested vote from peers c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:58:56.163414 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:56.164016 18614 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:56.164291 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e82471468e430b877403ce304eac78" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:56.164880 18468 raft_consensus.cc:3049] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:58:56.165169 18468 raft_consensus.cc:738] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2f9ce9a0415c4546b9a764ac06cefb38, State: Running, Role: LEADER
I20250112 20:58:56.165858 18468 consensus_queue.cc:260] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:56.166952 18468 raft_consensus.cc:3054] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:56.170043 18614 raft_consensus.cc:2463] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 3.
I20250112 20:58:56.170994 19242 leader_election.cc:304] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:56.171746 19428 raft_consensus.cc:2798] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:56.172099 19428 raft_consensus.cc:695] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:56.172645 18468 raft_consensus.cc:2463] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 3.
I20250112 20:58:56.172854 19428 consensus_queue.cc:237] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:58:56.180975 18290 catalog_manager.cc:5526] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 2 to 3, leader changed from 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 3 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } }
I20250112 20:58:56.239888 18468 raft_consensus.cc:1270] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:56.243319 19466 consensus_queue.cc:1035] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20250112 20:58:56.258791 18541 raft_consensus.cc:1270] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:56.259933 19466 consensus_queue.cc:1035] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:56.293670 18466 raft_consensus.cc:1270] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 4 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20250112 20:58:56.294605 19428 consensus_queue.cc:1035] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:56.306659 18541 raft_consensus.cc:1270] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 4 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20250112 20:58:56.307760 19426 consensus_queue.cc:1035] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:56.364698 18614 raft_consensus.cc:1270] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:56.365792 19426 consensus_queue.cc:1035] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:56.373391 18466 raft_consensus.cc:1270] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:56.375144 19425 consensus_queue.cc:1035] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:56.444192 18466 raft_consensus.cc:1270] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:56.445339 19425 consensus_queue.cc:1035] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:56.453979 18617 raft_consensus.cc:1270] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:56.457597 19428 consensus_queue.cc:1035] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:56.504628 18541 raft_consensus.cc:1270] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:56.505774 19428 consensus_queue.cc:1035] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:56.515908 18466 raft_consensus.cc:1270] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:56.520678 19428 consensus_queue.cc:1035] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:56.605538 18617 raft_consensus.cc:1270] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 3 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:56.606685 19428 consensus_queue.cc:1035] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:56.618068 18466 raft_consensus.cc:1270] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 3 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:56.619597 19426 consensus_queue.cc:1035] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:56.818665 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:56.854679 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:56.855109 18545 raft_consensus.cc:604] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:56.857300 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:56.857728 18545 raft_consensus.cc:604] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:56.859596 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a5ee370770b54ae593e07378574861d1"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:56.859959 18545 raft_consensus.cc:604] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:56.861668 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:56.862044 18545 raft_consensus.cc:604] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Received request to transfer leadership to TS c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:56.864012 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fe90092ab277453fb41e05a8be798304"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:56.864440 18545 raft_consensus.cc:604] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Received request to transfer leadership to TS c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:56.865248 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20250112 20:58:56.866366 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:56.866808 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:56.870997 19400 raft_consensus.cc:988] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:56.871394 19413 raft_consensus.cc:1076] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:56.872555 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52612
I20250112 20:58:56.873067 19292 raft_consensus.cc:491] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:56.873370 19292 raft_consensus.cc:3054] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:56.879367 19292 raft_consensus.cc:513] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:56.881778 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:56.883133 18545 raft_consensus.cc:3049] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:56.883452 18545 raft_consensus.cc:738] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:56.884184 18545 consensus_queue.cc:260] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:56.885284 18545 raft_consensus.cc:3054] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:56.888110 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b0cf559505640aea4edf6feedf96817" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:56.889838 18614 raft_consensus.cc:3054] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:56.898913 18614 raft_consensus.cc:2463] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:56.900051 19242 leader_election.cc:304] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:56.900969 19448 raft_consensus.cc:2798] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:56.899739 18545 raft_consensus.cc:2463] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:56.902544 19292 leader_election.cc:290] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:56.901377 19448 raft_consensus.cc:695] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:56.903618 19448 consensus_queue.cc:237] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:56.916033 18293 catalog_manager.cc:5526] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:56.929162 19402 raft_consensus.cc:988] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:56.929576 19413 raft_consensus.cc:1076] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Signalling peer c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:56.930644 18617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78"
dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:43602
I20250112 20:58:56.931089 18617 raft_consensus.cc:491] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:56.931330 18617 raft_consensus.cc:3054] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:56.935906 18617 raft_consensus.cc:513] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:56.937121 18617 leader_election.cc:290] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:56.937711 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:56.937906 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e73763ba5b1a4f59901b9e8f5eb6ab78" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:56.938273 18468 raft_consensus.cc:3054] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:56.938388 18545 raft_consensus.cc:3049] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:56.938690 18545 raft_consensus.cc:738] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:56.939204 18545 consensus_queue.cc:260] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:56.939988 18545 raft_consensus.cc:3054] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:56.943511 18468 raft_consensus.cc:2463] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:56.944625 18585 leader_election.cc:304] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:56.944958 18545 raft_consensus.cc:2463] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:56.945379 19466 raft_consensus.cc:2798] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:56.945858 19466 raft_consensus.cc:695] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:56.946630 19466 consensus_queue.cc:237] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:56.958734 18293 catalog_manager.cc:5526] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:57.017678 19400 raft_consensus.cc:988] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:57.018064 19402 raft_consensus.cc:1076] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:57.020028 19287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52612
I20250112 20:58:57.020726 19287 raft_consensus.cc:491] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:57.021080 19287 raft_consensus.cc:3054] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.028138 19287 raft_consensus.cc:513] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:57.030036 19287 leader_election.cc:290] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:57.030493 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:57.030854 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bed5fc535ff4870870d2b8f20d637d7" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:57.031080 18545 raft_consensus.cc:3049] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:57.031359 18545 raft_consensus.cc:738] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:57.031387 18617 raft_consensus.cc:3054] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.031924 18545 consensus_queue.cc:260] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:57.032799 18545 raft_consensus.cc:3054] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.035537 18617 raft_consensus.cc:2463] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:57.036710 19242 leader_election.cc:304] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:57.037482 19448 raft_consensus.cc:2798] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:57.037927 19448 raft_consensus.cc:695] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:57.038089 18545 raft_consensus.cc:2463] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:57.038800 19448 consensus_queue.cc:237] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:57.047005 18293 catalog_manager.cc:5526] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:57.068986 19490 raft_consensus.cc:988] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:57.069496 19402 raft_consensus.cc:1076] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Signalling peer c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:57.070591 18617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe90092ab277453fb41e05a8be798304"
dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:43602
I20250112 20:58:57.070978 18617 raft_consensus.cc:491] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:57.071202 18617 raft_consensus.cc:3054] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.075201 18617 raft_consensus.cc:513] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:57.076396 18617 leader_election.cc:290] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:57.077092 18468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe90092ab277453fb41e05a8be798304" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:57.077338 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe90092ab277453fb41e05a8be798304" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:57.077736 18468 raft_consensus.cc:3054] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.077849 18545 raft_consensus.cc:3049] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:57.078975 18545 raft_consensus.cc:738] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:57.079936 18545 consensus_queue.cc:260] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:57.080893 18545 raft_consensus.cc:3054] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.082855 18468 raft_consensus.cc:2463] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:57.083762 18585 leader_election.cc:304] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9ce9a0415c4546b9a764ac06cefb38, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:57.084412 19466 raft_consensus.cc:2798] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:57.084870 19466 raft_consensus.cc:695] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:57.085762 18545 raft_consensus.cc:2463] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:57.085611 19466 consensus_queue.cc:237] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:57.094353 18293 catalog_manager.cc:5526] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:57.262064 19490 raft_consensus.cc:988] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:57.262459 19402 raft_consensus.cc:1076] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:57.263538 19287 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ee370770b54ae593e07378574861d1"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52612
I20250112 20:58:57.264068 19287 raft_consensus.cc:491] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:57.264443 19287 raft_consensus.cc:3054] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.270059 19287 raft_consensus.cc:513] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:57.272003 19287 leader_election.cc:290] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:57.272207 18545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ee370770b54ae593e07378574861d1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:57.272481 18617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ee370770b54ae593e07378574861d1" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:57.272727 18545 raft_consensus.cc:3049] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:57.273056 18545 raft_consensus.cc:738] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:57.273065 18617 raft_consensus.cc:3054] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.273639 18545 consensus_queue.cc:260] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:57.274576 18545 raft_consensus.cc:3054] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.277247 18617 raft_consensus.cc:2463] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:57.278039 19242 leader_election.cc:304] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:57.278661 19448 raft_consensus.cc:2798] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:57.278657 18545 raft_consensus.cc:2463] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 2.
I20250112 20:58:57.279104 19448 raft_consensus.cc:695] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:57.279839 19448 consensus_queue.cc:237] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:57.288525 18293 catalog_manager.cc:5526] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 1 to 2, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:57.331060 18541 raft_consensus.cc:1270] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.332041 19425 consensus_queue.cc:1035] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.340991 18617 raft_consensus.cc:1270] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.342375 19448 consensus_queue.cc:1035] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.379555 18545 raft_consensus.cc:1270] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.380522 19466 consensus_queue.cc:1035] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.388319 18466 raft_consensus.cc:1270] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.390441 19466 consensus_queue.cc:1035] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.482105 18545 raft_consensus.cc:1270] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.483143 19466 consensus_queue.cc:1035] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.489274 18617 raft_consensus.cc:1270] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.490468 19448 consensus_queue.cc:1035] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.493707 18468 raft_consensus.cc:1270] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.494828 19466 consensus_queue.cc:1035] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.510932 18545 raft_consensus.cc:1270] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.512869 19426 consensus_queue.cc:1035] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.750439 18545 raft_consensus.cc:1270] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.751425 19425 consensus_queue.cc:1035] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.759354 18614 raft_consensus.cc:1270] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:57.761034 19425 consensus_queue.cc:1035] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:57.917511 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:57.949728 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:57.950345 18545 raft_consensus.cc:604] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Received request to transfer leadership to TS c60c0cbe377c450297e3d536b7f7c89c
I20250112 20:58:57.952677 18545 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693"
dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
mode: GRACEFUL
new_leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:39224
I20250112 20:58:57.953164 18545 raft_consensus.cc:604] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Received request to transfer leadership to TS a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:58:57.953994 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:58:57.955432 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:57.955998 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:57.958488 19490 raft_consensus.cc:988] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:57.958896 19490 raft_consensus.cc:1076] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Signalling peer c60c0cbe377c450297e3d536b7f7c89c to start an election
I20250112 20:58:57.960750 18617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e"
dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
from {username='slave'} at 127.0.0.1:43602
I20250112 20:58:57.961228 18617 raft_consensus.cc:491] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:57.961630 18617 raft_consensus.cc:3054] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.972162 18617 raft_consensus.cc:513] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:57.978283 18617 leader_election.cc:290] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Requested vote from peers 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469), 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861)
I20250112 20:58:57.978798 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:57.978688 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "161a73359f3a4bb19a4df8ad30c6669e" candidate_uuid: "c60c0cbe377c450297e3d536b7f7c89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2f9ce9a0415c4546b9a764ac06cefb38"
I20250112 20:58:57.979424 18543 raft_consensus.cc:3049] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:58:57.979754 18543 raft_consensus.cc:738] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:57.979794 18466 raft_consensus.cc:3054] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.980521 18543 consensus_queue.cc:260] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:57.981731 18543 raft_consensus.cc:3054] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:58:57.987188 18543 raft_consensus.cc:2463] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:57.987304 18466 raft_consensus.cc:2463] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c60c0cbe377c450297e3d536b7f7c89c in term 2.
I20250112 20:58:57.988112 18584 leader_election.cc:304] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88ef62b247304e079788e24ddfdaa4fb, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:57.989405 19466 raft_consensus.cc:2798] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:58:57.989781 19466 raft_consensus.cc:695] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Becoming Leader. State: Replica: c60c0cbe377c450297e3d536b7f7c89c, State: Running, Role: LEADER
I20250112 20:58:57.990548 19466 consensus_queue.cc:237] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } }
I20250112 20:58:58.001634 18293 catalog_manager.cc:5526] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c reported cstate change: term changed from 1 to 2, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132). New cstate: current_term: 2 leader_uuid: "c60c0cbe377c450297e3d536b7f7c89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:58.099750 19402 raft_consensus.cc:988] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb: : Instructing follower a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:58.100159 19402 raft_consensus.cc:1076] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Signalling peer a45f68dd3d9043809f3bb06287f5d2d3 to start an election
I20250112 20:58:58.101411 19292 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693"
dest_uuid: "a45f68dd3d9043809f3bb06287f5d2d3"
from {username='slave'} at 127.0.0.1:52612
I20250112 20:58:58.101900 19292 raft_consensus.cc:491] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:58:58.102149 19292 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:58.106280 19292 raft_consensus.cc:513] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:58.107748 19292 leader_election.cc:290] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 3 election: Requested vote from peers 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861), c60c0cbe377c450297e3d536b7f7c89c (127.15.23.132:40357)
I20250112 20:58:58.108517 18614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c60c0cbe377c450297e3d536b7f7c89c"
I20250112 20:58:58.109077 18614 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:58.109328 18543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb4e64edf27d4e91aca4fba312aff693" candidate_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "88ef62b247304e079788e24ddfdaa4fb"
I20250112 20:58:58.109882 18543 raft_consensus.cc:3049] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:58:58.110138 18543 raft_consensus.cc:738] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88ef62b247304e079788e24ddfdaa4fb, State: Running, Role: LEADER
I20250112 20:58:58.110649 18543 consensus_queue.cc:260] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:58.111430 18543 raft_consensus.cc:3054] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:58:58.113173 18614 raft_consensus.cc:2463] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 3.
I20250112 20:58:58.114040 19242 leader_election.cc:304] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f68dd3d9043809f3bb06287f5d2d3, c60c0cbe377c450297e3d536b7f7c89c; no voters:
I20250112 20:58:58.114801 19425 raft_consensus.cc:2798] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:58:58.115162 19425 raft_consensus.cc:695] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 LEADER]: Becoming Leader. State: Replica: a45f68dd3d9043809f3bb06287f5d2d3, State: Running, Role: LEADER
I20250112 20:58:58.115772 18543 raft_consensus.cc:2463] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a45f68dd3d9043809f3bb06287f5d2d3 in term 3.
I20250112 20:58:58.115855 19425 consensus_queue.cc:237] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } }
I20250112 20:58:58.125025 18293 catalog_manager.cc:5526] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 reported cstate change: term changed from 2 to 3, leader changed from 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131) to a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129). New cstate: current_term: 3 leader_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } health_report { overall_health: HEALTHY } } }
I20250112 20:58:58.542788 18543 raft_consensus.cc:1270] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:58.543773 19466 consensus_queue.cc:1035] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:58.555430 18466 raft_consensus.cc:1270] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Refusing update from remote peer c60c0cbe377c450297e3d536b7f7c89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:58:58.556504 19466 consensus_queue.cc:1035] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:58:58.706992 18614 raft_consensus.cc:1270] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 3 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:58.708019 19425 consensus_queue.cc:1035] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:58.710222 18545 raft_consensus.cc:1270] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 3 FOLLOWER]: Refusing update from remote peer a45f68dd3d9043809f3bb06287f5d2d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:58:58.711301 19448 consensus_queue.cc:1035] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88ef62b247304e079788e24ddfdaa4fb" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:58:59.000324 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:59.032577 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:58:59.033478 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:58:59.033931 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:59.063572 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:58:59.101747 15454 tablet_server.cc:178] TabletServer@127.15.23.129:38493 shutting down...
W20250112 20:58:59.130887 18583 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 222 similar messages]
I20250112 20:58:59.135658 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:58:59.138367 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.139791 15454 raft_consensus.cc:2238] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.140745 15454 raft_consensus.cc:2267] T 6f3629a889ed427db7d163963dd13507 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.140899 18583 consensus_peers.cc:487] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.143077 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.143882 15454 raft_consensus.cc:2238] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.144645 15454 raft_consensus.cc:2267] T d45830b149244e2f8bec3cac5228eb77 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.147084 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.147711 15454 raft_consensus.cc:2238] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.148037 15454 raft_consensus.cc:2267] T 985f9b21969e48739677e3cdc86e25de P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.150245 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.150871 15454 raft_consensus.cc:2238] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.151311 15454 raft_consensus.cc:2267] T eaf48f6743cb46e686eaa798a4a22957 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.153610 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.154410 15454 raft_consensus.cc:2238] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.155149 15454 raft_consensus.cc:2267] T 60856bd5ad724e3792e927130b6315cf P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.157227 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.157788 15454 raft_consensus.cc:2238] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.158267 15454 raft_consensus.cc:2267] T 3f52ac1efd0e417393e5127713a43a18 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.160493 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.161003 15454 raft_consensus.cc:2238] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.161383 15454 raft_consensus.cc:2267] T 3fa320ac4d374bcc8a1af58dc71bcdfe P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.163749 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.164251 15454 raft_consensus.cc:2238] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.165074 15454 raft_consensus.cc:2267] T fb4e64edf27d4e91aca4fba312aff693 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.167886 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.168828 15454 raft_consensus.cc:2238] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.169376 15454 raft_consensus.cc:2267] T 8db962a176db442490ad2e6133babb67 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.172154 18435 consensus_peers.cc:487] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.172881 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.173647 15454 raft_consensus.cc:2238] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.174285 15454 raft_consensus.cc:2267] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.176282 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.176863 15454 raft_consensus.cc:2238] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.177280 15454 raft_consensus.cc:2267] T c99e10fcc8944b44aff69f16d268a07c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.177875 18509 consensus_peers.cc:487] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.179096 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.179677 15454 raft_consensus.cc:2238] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.180300 15454 raft_consensus.cc:2267] T 6ca1ca0f378247598c915c11faee8ed2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.182022 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.182518 15454 raft_consensus.cc:2238] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.182888 15454 raft_consensus.cc:2267] T 1332a302b6a44f9c8963dfed69c078a5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.184494 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.185091 15454 raft_consensus.cc:2238] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.185428 15454 raft_consensus.cc:2267] T f4bdf5e8ae124786b98df6013ed28460 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.187286 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.187950 15454 raft_consensus.cc:2238] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.188349 15454 raft_consensus.cc:2267] T afe68bc3c3a54ebb816b85651cfd9d08 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.190066 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.190577 15454 raft_consensus.cc:2238] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.190933 15454 raft_consensus.cc:2267] T 1b92c85eeea5465ea50fa57eb9214098 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.192574 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.193071 15454 raft_consensus.cc:2238] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.193387 15454 raft_consensus.cc:2267] T b7f8bbae0e0c494ab580a5f2338ef4ea P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.195428 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.195952 15454 raft_consensus.cc:2238] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.196350 15454 raft_consensus.cc:2267] T 4295797580d643d88c1ef2701a123bef P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.198076 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.198558 15454 raft_consensus.cc:2238] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.198941 15454 raft_consensus.cc:2267] T f8820dfc60bc477985f861406c22dd3b P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.200790 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.201501 15454 raft_consensus.cc:2238] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.202304 15454 raft_consensus.cc:2267] T a5ee370770b54ae593e07378574861d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.204062 18583 consensus_peers.cc:487] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.204519 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.205098 15454 raft_consensus.cc:2238] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.205785 15454 raft_consensus.cc:2267] T 08844b71d8c54556af7a5d625ed646c2 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.207406 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.207865 15454 raft_consensus.cc:2238] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.208163 15454 raft_consensus.cc:2267] T 013cadee88a64ffe9b1b31198139d6d1 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.210054 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.210493 15454 raft_consensus.cc:2238] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.210806 15454 raft_consensus.cc:2267] T 9e59ecf931f041feba70ee5460de09da P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.212437 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.212935 15454 raft_consensus.cc:2238] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.213232 15454 raft_consensus.cc:2267] T c2f492fad2f5422392d1d141e7534579 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.214900 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.215399 15454 raft_consensus.cc:2238] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.215703 15454 raft_consensus.cc:2267] T 737c8961a9424363b63a9688853bea1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.217268 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.217736 15454 raft_consensus.cc:2238] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.218354 15454 raft_consensus.cc:2267] T 4bed5fc535ff4870870d2b8f20d637d7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.219954 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.220559 15454 raft_consensus.cc:2238] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.221004 15454 raft_consensus.cc:2267] T 3d939684f7a845f7b267cf420779b603 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.223074 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.223775 15454 raft_consensus.cc:2238] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.224239 15454 raft_consensus.cc:2267] T 6df407527b0d446795e620dec471e027 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.226465 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.226941 15454 raft_consensus.cc:2238] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.227269 15454 raft_consensus.cc:2267] T ee08d81c32b648e085f17dd937f5c512 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.228840 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.229293 15454 raft_consensus.cc:2238] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.229728 15454 raft_consensus.cc:2267] T f59cd634f5244017b18a111892243f57 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.231380 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.231961 15454 raft_consensus.cc:2238] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 4 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.232775 15454 raft_consensus.cc:2267] T 668ccd5a01ca4d4a85a4b033d0dcba06 P a45f68dd3d9043809f3bb06287f5d2d3 [term 4 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.234709 18509 consensus_peers.cc:487] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.234894 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.235512 15454 raft_consensus.cc:2238] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.236153 15454 raft_consensus.cc:2267] T 4b0cf559505640aea4edf6feedf96817 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.237896 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.238339 15454 raft_consensus.cc:2238] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.238698 15454 raft_consensus.cc:2267] T cdbd6c5512664ac7a36a13bd1a89d75f P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.240341 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.240831 15454 raft_consensus.cc:2238] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.241220 15454 raft_consensus.cc:2267] T af0c89ea5b0d4655959ccaad0b838bf4 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.243163 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.243723 15454 raft_consensus.cc:2238] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:59.243968 18509 consensus_peers.cc:487] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.244235 15454 raft_consensus.cc:2267] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.246039 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.246477 15454 raft_consensus.cc:2238] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.246778 15454 raft_consensus.cc:2267] T e1f0a2bc717d404f8f3afd03c3d5a60c P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.248526 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.249137 15454 raft_consensus.cc:2238] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.249733 15454 raft_consensus.cc:2267] T 95e82471468e430b877403ce304eac78 P a45f68dd3d9043809f3bb06287f5d2d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.251683 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.252295 15454 raft_consensus.cc:2238] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.252797 15454 raft_consensus.cc:2267] T 684a3c0a345f4ce483768dfe1572ca13 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.255173 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.255901 15454 raft_consensus.cc:2238] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.256371 15454 raft_consensus.cc:2267] T 6961f63ec790480aaa86f70431ed86c6 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.258603 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.259204 15454 raft_consensus.cc:2238] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.259676 15454 raft_consensus.cc:2267] T f7f1fd690e5d4207957654f28756dfaf P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.262060 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.262665 15454 raft_consensus.cc:2238] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.263025 15454 raft_consensus.cc:2267] T 7b6dc91797f14584b833e9dec80369f5 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.264861 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.265290 15454 raft_consensus.cc:2238] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.265615 15454 raft_consensus.cc:2267] T 16d5dc78c20741968c7ca8f7ecec1c07 P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.267480 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.268105 15454 raft_consensus.cc:2238] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.269047 15454 raft_consensus.cc:2267] T bb32432c5bc044b5b9f6871bbb30be64 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.270270 18435 consensus_peers.cc:487] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.271260 15454 tablet_replica.cc:331] stopping tablet replica
W20250112 20:58:59.271603 18583 consensus_peers.cc:487] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.271847 15454 raft_consensus.cc:2238] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.272243 15454 raft_consensus.cc:2267] T 219a31736df4499fb96cacfe05747d1a P a45f68dd3d9043809f3bb06287f5d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.273941 15454 tablet_replica.cc:331] stopping tablet replica
I20250112 20:58:59.274343 15454 raft_consensus.cc:2238] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.274998 15454 raft_consensus.cc:2267] T d7daf053c1144c6797f1ac89fedd0fd0 P a45f68dd3d9043809f3bb06287f5d2d3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.312812 18509 consensus_peers.cc:487] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.314260 18509 consensus_peers.cc:487] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.318099 18435 consensus_peers.cc:487] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.331017 18583 consensus_peers.cc:487] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.331445 18583 consensus_peers.cc:487] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.332123 18435 consensus_peers.cc:487] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.339370 15454 tablet_server.cc:195] TabletServer@127.15.23.129:38493 shutdown complete.
W20250112 20:58:59.344980 18435 consensus_peers.cc:487] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.347087 18435 consensus_peers.cc:487] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.367467 18509 consensus_peers.cc:487] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.380299 18509 consensus_peers.cc:487] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.406845 18583 consensus_peers.cc:487] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.410180 15454 tablet_server.cc:178] TabletServer@127.15.23.130:0 shutting down...
W20250112 20:58:59.427822 18583 consensus_peers.cc:487] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.434885 18435 consensus_peers.cc:487] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.446323 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:58:59.447144 15454 tablet_replica.cc:331] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.447769 15454 raft_consensus.cc:2238] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.448606 15454 raft_consensus.cc:2267] T f4bdf5e8ae124786b98df6013ed28460 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.450367 15454 tablet_replica.cc:331] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.450850 15454 raft_consensus.cc:2238] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.451187 15454 raft_consensus.cc:2267] T afe68bc3c3a54ebb816b85651cfd9d08 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.452795 15454 tablet_replica.cc:331] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.453354 15454 raft_consensus.cc:2238] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.453708 15454 raft_consensus.cc:2267] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.456095 15454 tablet_replica.cc:331] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.456866 15454 raft_consensus.cc:2238] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.457345 15454 raft_consensus.cc:2267] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.458412 18435 consensus_peers.cc:487] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.459721 15454 tablet_replica.cc:331] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.460666 15454 raft_consensus.cc:2238] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.461155 15454 raft_consensus.cc:2267] T 6db2806263c3471cac0496b194a73ab9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.461213 18585 consensus_peers.cc:487] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.463626 15454 tablet_replica.cc:331] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
W20250112 20:58:59.463689 18509 consensus_peers.cc:487] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.464097 18435 consensus_peers.cc:487] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.464377 15454 raft_consensus.cc:2238] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.465250 15454 raft_consensus.cc:2267] T cdbd6c5512664ac7a36a13bd1a89d75f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.467236 15454 tablet_replica.cc:331] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.467929 15454 raft_consensus.cc:2238] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:59.467943 18509 consensus_peers.cc:487] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.468335 15454 raft_consensus.cc:2267] T e8704b5088604084a641aa00baf84be8 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.470016 15454 tablet_replica.cc:331] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.470481 15454 raft_consensus.cc:2238] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.470834 15454 raft_consensus.cc:2267] T 985f9b21969e48739677e3cdc86e25de P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.472316 15454 tablet_replica.cc:331] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.472823 15454 raft_consensus.cc:2238] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.473136 15454 raft_consensus.cc:2267] T 161a73359f3a4bb19a4df8ad30c6669e P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.474834 15454 tablet_replica.cc:331] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.475332 15454 raft_consensus.cc:2238] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.475733 15454 raft_consensus.cc:2267] T 3fa320ac4d374bcc8a1af58dc71bcdfe P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.477190 15454 tablet_replica.cc:331] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.477630 15454 raft_consensus.cc:2238] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.477962 15454 raft_consensus.cc:2267] T fe90092ab277453fb41e05a8be798304 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.479393 15454 tablet_replica.cc:331] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.479866 15454 raft_consensus.cc:2238] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.480518 15454 raft_consensus.cc:2267] T 759cdb36c54b4950b89e7502ae9e878f P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.481981 15454 tablet_replica.cc:331] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.482398 15454 raft_consensus.cc:2238] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.483033 15454 raft_consensus.cc:2267] T 1332a302b6a44f9c8963dfed69c078a5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.484787 15454 tablet_replica.cc:331] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.485394 15454 raft_consensus.cc:2238] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.485780 15454 raft_consensus.cc:2267] T 6ca1ca0f378247598c915c11faee8ed2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.487565 18509 consensus_peers.cc:487] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.487727 15454 tablet_replica.cc:331] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.488294 15454 raft_consensus.cc:2238] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.489014 15454 raft_consensus.cc:2267] T c99e10fcc8944b44aff69f16d268a07c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.490648 15454 tablet_replica.cc:331] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.491182 15454 raft_consensus.cc:2238] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.491509 15454 raft_consensus.cc:2267] T 08844b71d8c54556af7a5d625ed646c2 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.493031 15454 tablet_replica.cc:331] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.493463 15454 raft_consensus.cc:2238] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.493794 15454 raft_consensus.cc:2267] T 8300b885639841769a76417fb45f2db7 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.495373 15454 tablet_replica.cc:331] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.495821 15454 raft_consensus.cc:2238] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.496157 15454 raft_consensus.cc:2267] T c2f492fad2f5422392d1d141e7534579 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.498291 15454 tablet_replica.cc:331] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.499008 15454 raft_consensus.cc:2238] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.499825 15454 raft_consensus.cc:2267] T 737c8961a9424363b63a9688853bea1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.501837 15454 tablet_replica.cc:331] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.502460 15454 raft_consensus.cc:2238] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:59.502581 18435 consensus_peers.cc:487] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.502970 15454 raft_consensus.cc:2267] T eeb9028573e647bba81035b9c4a15146 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.504972 15454 tablet_replica.cc:331] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
W20250112 20:58:59.505522 18509 consensus_peers.cc:487] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.505527 15454 raft_consensus.cc:2238] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.506218 15454 raft_consensus.cc:2267] T 6f3629a889ed427db7d163963dd13507 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.507856 15454 tablet_replica.cc:331] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.508554 15454 raft_consensus.cc:2238] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.509406 15454 raft_consensus.cc:2267] T 3bea0704826246c1b1e2901dbab0cb15 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.510905 15454 tablet_replica.cc:331] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.511358 15454 raft_consensus.cc:2238] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.512147 15454 raft_consensus.cc:2267] T 219a31736df4499fb96cacfe05747d1a P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.513904 15454 tablet_replica.cc:331] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
W20250112 20:58:59.514125 18511 consensus_peers.cc:487] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.514454 15454 raft_consensus.cc:2238] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.514922 15454 raft_consensus.cc:2267] T bb32432c5bc044b5b9f6871bbb30be64 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.517078 15454 tablet_replica.cc:331] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.517570 15454 raft_consensus.cc:2238] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.517906 15454 raft_consensus.cc:2267] T 6961f63ec790480aaa86f70431ed86c6 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.519608 15454 tablet_replica.cc:331] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.520076 15454 raft_consensus.cc:2238] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.520519 15454 raft_consensus.cc:2267] T 5547898f3d53475fb47696f4ddd6ef1c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.523068 15454 tablet_replica.cc:331] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.523816 15454 raft_consensus.cc:2238] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.524621 15454 raft_consensus.cc:2267] T c257b2d0772f4078b2ca2d7c1298e4ae P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.526508 18583 consensus_peers.cc:487] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c -> Peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Couldn't send request to peer a45f68dd3d9043809f3bb06287f5d2d3. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.526943 15454 tablet_replica.cc:331] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.527494 15454 raft_consensus.cc:2238] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.528129 15454 raft_consensus.cc:2267] T f7f1fd690e5d4207957654f28756dfaf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.530371 15454 tablet_replica.cc:331] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.530851 15454 raft_consensus.cc:2238] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.531193 15454 raft_consensus.cc:2267] T 7b6dc91797f14584b833e9dec80369f5 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.532742 15454 tablet_replica.cc:331] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.533196 15454 raft_consensus.cc:2238] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.533821 15454 raft_consensus.cc:2267] T 4295797580d643d88c1ef2701a123bef P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.535844 15454 tablet_replica.cc:331] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.536363 15454 raft_consensus.cc:2238] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.536782 15454 raft_consensus.cc:2267] T 1b92c85eeea5465ea50fa57eb9214098 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.538543 15454 tablet_replica.cc:331] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.538996 15454 raft_consensus.cc:2238] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.539674 15454 raft_consensus.cc:2267] T b7f8bbae0e0c494ab580a5f2338ef4ea P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.541462 15454 tablet_replica.cc:331] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.541903 15454 raft_consensus.cc:2238] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.542194 15454 raft_consensus.cc:2267] T 95e82471468e430b877403ce304eac78 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.543664 15454 tablet_replica.cc:331] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.544204 15454 raft_consensus.cc:2238] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.545153 15454 raft_consensus.cc:2267] T e1f0a2bc717d404f8f3afd03c3d5a60c P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.547518 15454 tablet_replica.cc:331] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.548224 15454 raft_consensus.cc:2238] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.549155 15454 raft_consensus.cc:2267] T ee08d81c32b648e085f17dd937f5c512 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.551498 15454 tablet_replica.cc:331] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
W20250112 20:58:59.552047 18511 consensus_peers.cc:487] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.552224 15454 raft_consensus.cc:2238] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.552848 15454 raft_consensus.cc:2267] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.555159 15454 tablet_replica.cc:331] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.555891 15454 raft_consensus.cc:2238] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.556329 15454 raft_consensus.cc:2267] T ef6e68976f1b46278833369d2b5c0e95 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.557904 15454 tablet_replica.cc:331] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.558360 15454 raft_consensus.cc:2238] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.558749 15454 raft_consensus.cc:2267] T 5ec5696903d64121bab105da9eb5a3b9 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.560576 15454 tablet_replica.cc:331] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.561224 15454 raft_consensus.cc:2238] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:59.561514 18511 consensus_peers.cc:487] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.561663 15454 raft_consensus.cc:2267] T 6df407527b0d446795e620dec471e027 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.563313 15454 tablet_replica.cc:331] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.563772 15454 raft_consensus.cc:2238] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.564111 15454 raft_consensus.cc:2267] T d45830b149244e2f8bec3cac5228eb77 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.565765 15454 tablet_replica.cc:331] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.566211 15454 raft_consensus.cc:2238] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.566848 15454 raft_consensus.cc:2267] T f59cd634f5244017b18a111892243f57 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.568972 15454 tablet_replica.cc:331] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.569608 15454 raft_consensus.cc:2238] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.570106 15454 raft_consensus.cc:2267] T 9e59ecf931f041feba70ee5460de09da P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.571475 18585 consensus_peers.cc:487] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.572209 15454 tablet_replica.cc:331] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.572728 15454 raft_consensus.cc:2238] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.573091 15454 raft_consensus.cc:2267] T 60856bd5ad724e3792e927130b6315cf P 2f9ce9a0415c4546b9a764ac06cefb38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.574985 15454 tablet_replica.cc:331] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38: stopping tablet replica
I20250112 20:58:59.575465 15454 raft_consensus.cc:2238] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:59.575800 18585 consensus_peers.cc:487] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.575919 15454 raft_consensus.cc:2267] T 1906b96af18e4a7184907ae1b525aa02 P 2f9ce9a0415c4546b9a764ac06cefb38 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.614449 18585 consensus_peers.cc:487] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.615020 18585 consensus_peers.cc:487] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.637470 15454 tablet_server.cc:195] TabletServer@127.15.23.130:0 shutdown complete.
W20250112 20:58:59.652097 18511 consensus_peers.cc:487] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.654491 18585 consensus_peers.cc:487] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.660255 18511 consensus_peers.cc:487] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.672262 18511 consensus_peers.cc:487] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.678112 18511 consensus_peers.cc:487] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.708035 18585 consensus_peers.cc:487] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.710820 15454 tablet_server.cc:178] TabletServer@127.15.23.131:0 shutting down...
W20250112 20:58:59.742863 18585 consensus_peers.cc:487] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.748205 18585 consensus_peers.cc:487] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.756286 18511 consensus_peers.cc:487] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:58:59.757563 18511 consensus_peers.cc:487] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.760028 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:58:59.760732 15454 tablet_replica.cc:331] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.761255 15454 raft_consensus.cc:2238] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.761672 15454 raft_consensus.cc:2267] T f4bdf5e8ae124786b98df6013ed28460 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.763442 15454 tablet_replica.cc:331] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.764053 15454 raft_consensus.cc:2238] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.764802 15454 raft_consensus.cc:2267] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.767194 15454 tablet_replica.cc:331] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.767755 15454 raft_consensus.cc:2238] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.768095 15454 raft_consensus.cc:2267] T d7daf053c1144c6797f1ac89fedd0fd0 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.769694 15454 tablet_replica.cc:331] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.770170 15454 raft_consensus.cc:2238] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.770771 15454 raft_consensus.cc:2267] T f8820dfc60bc477985f861406c22dd3b P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.772239 15454 tablet_replica.cc:331] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.772734 15454 raft_consensus.cc:2238] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.773093 15454 raft_consensus.cc:2267] T e73763ba5b1a4f59901b9e8f5eb6ab78 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.774621 15454 tablet_replica.cc:331] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.775081 15454 raft_consensus.cc:2238] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.775410 15454 raft_consensus.cc:2267] T 6db2806263c3471cac0496b194a73ab9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.777235 15454 tablet_replica.cc:331] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.777889 15454 raft_consensus.cc:2238] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.778752 15454 raft_consensus.cc:2267] T 3f52ac1efd0e417393e5127713a43a18 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.780937 15454 tablet_replica.cc:331] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.781661 15454 raft_consensus.cc:2238] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.782115 15454 raft_consensus.cc:2267] T 161a73359f3a4bb19a4df8ad30c6669e P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.782379 18584 consensus_peers.cc:487] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.784231 15454 tablet_replica.cc:331] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.785102 15454 raft_consensus.cc:2238] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.785629 15454 raft_consensus.cc:2267] T fe90092ab277453fb41e05a8be798304 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.787863 15454 tablet_replica.cc:331] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.788697 15454 raft_consensus.cc:2238] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.789644 15454 raft_consensus.cc:2267] T 16d5dc78c20741968c7ca8f7ecec1c07 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.791559 15454 tablet_replica.cc:331] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.792058 15454 raft_consensus.cc:2238] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.792425 15454 raft_consensus.cc:2267] T 759cdb36c54b4950b89e7502ae9e878f P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.794091 15454 tablet_replica.cc:331] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.794567 15454 raft_consensus.cc:2238] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.794926 15454 raft_consensus.cc:2267] T 08844b71d8c54556af7a5d625ed646c2 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.796645 15454 tablet_replica.cc:331] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.797331 15454 raft_consensus.cc:2238] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.797748 15454 raft_consensus.cc:2267] T 8300b885639841769a76417fb45f2db7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.797858 18584 consensus_peers.cc:487] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.799330 15454 tablet_replica.cc:331] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
W20250112 20:58:59.799721 18584 consensus_peers.cc:487] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.799964 15454 raft_consensus.cc:2238] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.800405 15454 raft_consensus.cc:2267] T e8704b5088604084a641aa00baf84be8 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.802013 15454 tablet_replica.cc:331] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.802536 15454 raft_consensus.cc:2238] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.802896 15454 raft_consensus.cc:2267] T cdbd6c5512664ac7a36a13bd1a89d75f P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.804497 15454 tablet_replica.cc:331] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.805020 15454 raft_consensus.cc:2238] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.805696 15454 raft_consensus.cc:2267] T af0c89ea5b0d4655959ccaad0b838bf4 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.807309 15454 tablet_replica.cc:331] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.807865 15454 raft_consensus.cc:2238] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.808218 15454 raft_consensus.cc:2267] T fb4e64edf27d4e91aca4fba312aff693 P 88ef62b247304e079788e24ddfdaa4fb [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.809892 15454 tablet_replica.cc:331] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.810369 15454 raft_consensus.cc:2238] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.811030 15454 raft_consensus.cc:2267] T afe68bc3c3a54ebb816b85651cfd9d08 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.812692 15454 tablet_replica.cc:331] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.813172 15454 raft_consensus.cc:2238] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 4 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.813526 15454 raft_consensus.cc:2267] T 668ccd5a01ca4d4a85a4b033d0dcba06 P 88ef62b247304e079788e24ddfdaa4fb [term 4 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.815026 15454 tablet_replica.cc:331] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.815531 15454 raft_consensus.cc:2238] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.816182 15454 raft_consensus.cc:2267] T 5ec5696903d64121bab105da9eb5a3b9 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.817731 15454 tablet_replica.cc:331] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.818176 15454 raft_consensus.cc:2238] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.818517 15454 raft_consensus.cc:2267] T 4b0cf559505640aea4edf6feedf96817 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.820045 15454 tablet_replica.cc:331] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.820536 15454 raft_consensus.cc:2238] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.820945 15454 raft_consensus.cc:2267] T ef6e68976f1b46278833369d2b5c0e95 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.822537 15454 tablet_replica.cc:331] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.822973 15454 raft_consensus.cc:2238] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.823307 15454 raft_consensus.cc:2267] T ee08d81c32b648e085f17dd937f5c512 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.825071 15454 tablet_replica.cc:331] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.825618 15454 raft_consensus.cc:2238] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.826046 15454 raft_consensus.cc:2267] T 684a3c0a345f4ce483768dfe1572ca13 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.827771 15454 tablet_replica.cc:331] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.828236 15454 raft_consensus.cc:2238] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
W20250112 20:58:59.828298 18585 consensus_peers.cc:487] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.828656 15454 raft_consensus.cc:2267] T e1f0a2bc717d404f8f3afd03c3d5a60c P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.830415 15454 tablet_replica.cc:331] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.830909 15454 raft_consensus.cc:2238] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.831255 15454 raft_consensus.cc:2267] T 8db962a176db442490ad2e6133babb67 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.832898 15454 tablet_replica.cc:331] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.833411 15454 raft_consensus.cc:2238] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.834069 15454 raft_consensus.cc:2267] T 9e59ecf931f041feba70ee5460de09da P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.835865 15454 tablet_replica.cc:331] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.836570 15454 raft_consensus.cc:2238] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.837494 15454 raft_consensus.cc:2267] T 013cadee88a64ffe9b1b31198139d6d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.839772 15454 tablet_replica.cc:331] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.840282 15454 raft_consensus.cc:2238] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.840684 15454 raft_consensus.cc:2267] T c99e10fcc8944b44aff69f16d268a07c P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.842331 15454 tablet_replica.cc:331] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.842823 15454 raft_consensus.cc:2238] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.843240 15454 raft_consensus.cc:2267] T 4295797580d643d88c1ef2701a123bef P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.844969 15454 tablet_replica.cc:331] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.845453 15454 raft_consensus.cc:2238] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.845830 15454 raft_consensus.cc:2267] T b7f8bbae0e0c494ab580a5f2338ef4ea P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.847414 15454 tablet_replica.cc:331] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.847914 15454 raft_consensus.cc:2238] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.848896 15454 raft_consensus.cc:2267] T eeb9028573e647bba81035b9c4a15146 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.851223 15454 tablet_replica.cc:331] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.851914 15454 raft_consensus.cc:2238] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.852409 15454 raft_consensus.cc:2267] T 3bea0704826246c1b1e2901dbab0cb15 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.854705 15454 tablet_replica.cc:331] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.855381 15454 raft_consensus.cc:2238] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.855875 15454 raft_consensus.cc:2267] T 3d939684f7a845f7b267cf420779b603 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.856629 18585 consensus_peers.cc:487] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Couldn't send request to peer 2f9ce9a0415c4546b9a764ac06cefb38. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.857720 15454 tablet_replica.cc:331] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.858258 15454 raft_consensus.cc:2238] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.858884 15454 raft_consensus.cc:2267] T 7b6dc91797f14584b833e9dec80369f5 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.860451 15454 tablet_replica.cc:331] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.860924 15454 raft_consensus.cc:2238] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.861271 15454 raft_consensus.cc:2267] T c257b2d0772f4078b2ca2d7c1298e4ae P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.862768 15454 tablet_replica.cc:331] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.863204 15454 raft_consensus.cc:2238] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.864130 15454 raft_consensus.cc:2267] T 5547898f3d53475fb47696f4ddd6ef1c P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.866510 15454 tablet_replica.cc:331] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.867152 15454 raft_consensus.cc:2238] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.867944 15454 raft_consensus.cc:2267] T 985f9b21969e48739677e3cdc86e25de P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.870164 15454 tablet_replica.cc:331] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.870821 15454 raft_consensus.cc:2238] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.871258 15454 raft_consensus.cc:2267] T eaf48f6743cb46e686eaa798a4a22957 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.873518 15454 tablet_replica.cc:331] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.873991 15454 raft_consensus.cc:2238] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.874428 15454 raft_consensus.cc:2267] T 60856bd5ad724e3792e927130b6315cf P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.876149 15454 tablet_replica.cc:331] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.876631 15454 raft_consensus.cc:2238] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
W20250112 20:58:59.876791 18584 consensus_peers.cc:487] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.877310 15454 raft_consensus.cc:2267] T 1906b96af18e4a7184907ae1b525aa02 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.879154 15454 tablet_replica.cc:331] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.879596 15454 raft_consensus.cc:2238] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.879927 15454 raft_consensus.cc:2267] T 4bed5fc535ff4870870d2b8f20d637d7 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:58:59.881592 15454 tablet_replica.cc:331] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.882197 15454 raft_consensus.cc:2238] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:58:59.882892 15454 raft_consensus.cc:2267] T c2f492fad2f5422392d1d141e7534579 P 88ef62b247304e079788e24ddfdaa4fb [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.884184 18584 consensus_peers.cc:487] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.884771 15454 tablet_replica.cc:331] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb: stopping tablet replica
I20250112 20:58:59.885284 15454 raft_consensus.cc:2238] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:58:59.885651 15454 raft_consensus.cc:2267] T a5ee370770b54ae593e07378574861d1 P 88ef62b247304e079788e24ddfdaa4fb [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:58:59.966722 18584 consensus_peers.cc:487] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:39861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:58:59.975510 15454 tablet_server.cc:195] TabletServer@127.15.23.131:0 shutdown complete.
W20250112 20:58:59.989529 18584 consensus_peers.cc:487] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:39861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:59:00.015450 18584 consensus_peers.cc:487] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:39861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:00.042124 15454 tablet_server.cc:178] TabletServer@127.15.23.132:0 shutting down...
W20250112 20:59:00.073372 18584 consensus_peers.cc:487] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c -> Peer 88ef62b247304e079788e24ddfdaa4fb (127.15.23.131:39861): Couldn't send request to peer 88ef62b247304e079788e24ddfdaa4fb. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.131:39861: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:00.082551 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:59:00.083222 15454 tablet_replica.cc:331] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.083796 15454 raft_consensus.cc:2238] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.084200 15454 raft_consensus.cc:2267] T 3fc3d5bd7d04459c9c1e7ccb28eb2ee7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.086079 15454 tablet_replica.cc:331] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.086633 15454 raft_consensus.cc:2238] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.087321 15454 raft_consensus.cc:2267] T fe90092ab277453fb41e05a8be798304 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.088979 15454 tablet_replica.cc:331] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.089466 15454 raft_consensus.cc:2238] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.090080 15454 raft_consensus.cc:2267] T 3fa320ac4d374bcc8a1af58dc71bcdfe P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.091588 15454 tablet_replica.cc:331] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.092078 15454 raft_consensus.cc:2238] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.092418 15454 raft_consensus.cc:2267] T f8820dfc60bc477985f861406c22dd3b P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.094020 15454 tablet_replica.cc:331] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.094535 15454 raft_consensus.cc:2238] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.094904 15454 raft_consensus.cc:2267] T 6ca1ca0f378247598c915c11faee8ed2 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.096489 15454 tablet_replica.cc:331] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.096993 15454 raft_consensus.cc:2238] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.097615 15454 raft_consensus.cc:2267] T 161a73359f3a4bb19a4df8ad30c6669e P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.099093 15454 tablet_replica.cc:331] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.099583 15454 raft_consensus.cc:2238] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.099937 15454 raft_consensus.cc:2267] T 16d5dc78c20741968c7ca8f7ecec1c07 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.101675 15454 tablet_replica.cc:331] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.102130 15454 raft_consensus.cc:2238] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.102487 15454 raft_consensus.cc:2267] T d7daf053c1144c6797f1ac89fedd0fd0 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.104023 15454 tablet_replica.cc:331] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.104468 15454 raft_consensus.cc:2238] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.105158 15454 raft_consensus.cc:2267] T e73763ba5b1a4f59901b9e8f5eb6ab78 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.106654 15454 tablet_replica.cc:331] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.107120 15454 raft_consensus.cc:2238] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.107741 15454 raft_consensus.cc:2267] T 6db2806263c3471cac0496b194a73ab9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.109344 15454 tablet_replica.cc:331] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.109838 15454 raft_consensus.cc:2238] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.110471 15454 raft_consensus.cc:2267] T e8704b5088604084a641aa00baf84be8 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.112087 15454 tablet_replica.cc:331] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.112586 15454 raft_consensus.cc:2238] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.112963 15454 raft_consensus.cc:2267] T af0c89ea5b0d4655959ccaad0b838bf4 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.114563 15454 tablet_replica.cc:331] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.115083 15454 raft_consensus.cc:2238] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.115442 15454 raft_consensus.cc:2267] T 3f52ac1efd0e417393e5127713a43a18 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.117081 15454 tablet_replica.cc:331] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.117604 15454 raft_consensus.cc:2238] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.118221 15454 raft_consensus.cc:2267] T 8300b885639841769a76417fb45f2db7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.119771 15454 tablet_replica.cc:331] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.120230 15454 raft_consensus.cc:2238] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.120571 15454 raft_consensus.cc:2267] T 759cdb36c54b4950b89e7502ae9e878f P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.122249 15454 tablet_replica.cc:331] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.122735 15454 raft_consensus.cc:2238] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.123109 15454 raft_consensus.cc:2267] T 013cadee88a64ffe9b1b31198139d6d1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.124531 15454 tablet_replica.cc:331] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.125073 15454 raft_consensus.cc:2238] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.125442 15454 raft_consensus.cc:2267] T 4bed5fc535ff4870870d2b8f20d637d7 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.126148 19519 raft_consensus.cc:491] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a45f68dd3d9043809f3bb06287f5d2d3)
I20250112 20:59:00.126518 19519 raft_consensus.cc:513] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c60c0cbe377c450297e3d536b7f7c89c" member_type: VOTER last_known_addr { host: "127.15.23.132" port: 40357 } } peers { permanent_uuid: "a45f68dd3d9043809f3bb06287f5d2d3" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 38493 } } peers { permanent_uuid: "2f9ce9a0415c4546b9a764ac06cefb38" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 35469 } }
I20250112 20:59:00.127197 15454 tablet_replica.cc:331] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.127842 15454 raft_consensus.cc:2238] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.128067 19519 leader_election.cc:290] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493), 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469)
I20250112 20:59:00.128309 15454 raft_consensus.cc:2267] T 737c8961a9424363b63a9688853bea1a P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.130772 15454 tablet_replica.cc:331] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.131510 15454 raft_consensus.cc:2238] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 3 FOLLOWER]: Raft consensus shutting down.
W20250112 20:59:00.131600 18583 leader_election.cc:336] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a45f68dd3d9043809f3bb06287f5d2d3 (127.15.23.129:38493): Network error: Client connection negotiation failed: client connection to 127.15.23.129:38493: connect: Connection refused (error 111)
I20250112 20:59:00.131906 15454 raft_consensus.cc:2267] T 95e82471468e430b877403ce304eac78 P c60c0cbe377c450297e3d536b7f7c89c [term 3 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:00.132861 18585 leader_election.cc:336] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2f9ce9a0415c4546b9a764ac06cefb38 (127.15.23.130:35469): Network error: Client connection negotiation failed: client connection to 127.15.23.130:35469: connect: Connection refused (error 111)
I20250112 20:59:00.133267 18585 leader_election.cc:304] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c60c0cbe377c450297e3d536b7f7c89c; no voters: 2f9ce9a0415c4546b9a764ac06cefb38, a45f68dd3d9043809f3bb06287f5d2d3
I20250112 20:59:00.133894 19519 raft_consensus.cc:2743] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250112 20:59:00.134308 15454 tablet_replica.cc:331] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.134884 15454 raft_consensus.cc:2238] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.135704 15454 raft_consensus.cc:2267] T 684a3c0a345f4ce483768dfe1572ca13 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.137571 15454 tablet_replica.cc:331] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.138036 15454 raft_consensus.cc:2238] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.138329 15454 raft_consensus.cc:2267] T 3bea0704826246c1b1e2901dbab0cb15 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.139755 15454 tablet_replica.cc:331] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.140193 15454 raft_consensus.cc:2238] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.140504 15454 raft_consensus.cc:2267] T 6f3629a889ed427db7d163963dd13507 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.142066 15454 tablet_replica.cc:331] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.142494 15454 raft_consensus.cc:2238] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.142796 15454 raft_consensus.cc:2267] T eeb9028573e647bba81035b9c4a15146 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.144371 15454 tablet_replica.cc:331] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.144865 15454 raft_consensus.cc:2238] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.145239 15454 raft_consensus.cc:2267] T fb4e64edf27d4e91aca4fba312aff693 P c60c0cbe377c450297e3d536b7f7c89c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.146761 15454 tablet_replica.cc:331] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.147212 15454 raft_consensus.cc:2238] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.147509 15454 raft_consensus.cc:2267] T 4b0cf559505640aea4edf6feedf96817 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.149055 15454 tablet_replica.cc:331] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.149479 15454 raft_consensus.cc:2238] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.149792 15454 raft_consensus.cc:2267] T 5ec5696903d64121bab105da9eb5a3b9 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.151300 15454 tablet_replica.cc:331] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.151737 15454 raft_consensus.cc:2238] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.152359 15454 raft_consensus.cc:2267] T ef6e68976f1b46278833369d2b5c0e95 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.153923 15454 tablet_replica.cc:331] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.154338 15454 raft_consensus.cc:2238] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.154953 15454 raft_consensus.cc:2267] T 6961f63ec790480aaa86f70431ed86c6 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.156508 15454 tablet_replica.cc:331] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.156973 15454 raft_consensus.cc:2238] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.157289 15454 raft_consensus.cc:2267] T f7f1fd690e5d4207957654f28756dfaf P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.158682 15454 tablet_replica.cc:331] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.159127 15454 raft_consensus.cc:2238] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.159422 15454 raft_consensus.cc:2267] T c257b2d0772f4078b2ca2d7c1298e4ae P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.161010 15454 tablet_replica.cc:331] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.161432 15454 raft_consensus.cc:2238] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.161762 15454 raft_consensus.cc:2267] T 5547898f3d53475fb47696f4ddd6ef1c P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.163200 15454 tablet_replica.cc:331] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.163589 15454 raft_consensus.cc:2238] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.164175 15454 raft_consensus.cc:2267] T eaf48f6743cb46e686eaa798a4a22957 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.165758 15454 tablet_replica.cc:331] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.166189 15454 raft_consensus.cc:2238] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.166496 15454 raft_consensus.cc:2267] T 1906b96af18e4a7184907ae1b525aa02 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.167999 15454 tablet_replica.cc:331] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.168421 15454 raft_consensus.cc:2238] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.168812 15454 raft_consensus.cc:2267] T 1332a302b6a44f9c8963dfed69c078a5 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.170377 15454 tablet_replica.cc:331] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.170840 15454 raft_consensus.cc:2238] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.171167 15454 raft_consensus.cc:2267] T 219a31736df4499fb96cacfe05747d1a P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.172677 15454 tablet_replica.cc:331] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.173131 15454 raft_consensus.cc:2238] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.173475 15454 raft_consensus.cc:2267] T bb32432c5bc044b5b9f6871bbb30be64 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.174949 15454 tablet_replica.cc:331] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.175405 15454 raft_consensus.cc:2238] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.175757 15454 raft_consensus.cc:2267] T 523dfa2a1c9e4cdf9def87c077e7c7f1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.177268 15454 tablet_replica.cc:331] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.177706 15454 raft_consensus.cc:2238] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.178368 15454 raft_consensus.cc:2267] T 8db962a176db442490ad2e6133babb67 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.180016 15454 tablet_replica.cc:331] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.180459 15454 raft_consensus.cc:2238] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.181125 15454 raft_consensus.cc:2267] T 6df407527b0d446795e620dec471e027 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.182683 15454 tablet_replica.cc:331] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.183149 15454 raft_consensus.cc:2238] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.183818 15454 raft_consensus.cc:2267] T 3d939684f7a845f7b267cf420779b603 P c60c0cbe377c450297e3d536b7f7c89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.185428 15454 tablet_replica.cc:331] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.185873 15454 raft_consensus.cc:2238] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.186234 15454 raft_consensus.cc:2267] T f59cd634f5244017b18a111892243f57 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.187822 15454 tablet_replica.cc:331] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.188226 15454 raft_consensus.cc:2238] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.188668 15454 raft_consensus.cc:2267] T d45830b149244e2f8bec3cac5228eb77 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.190171 15454 tablet_replica.cc:331] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.190582 15454 raft_consensus.cc:2238] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.191201 15454 raft_consensus.cc:2267] T 1b92c85eeea5465ea50fa57eb9214098 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.193490 15454 tablet_replica.cc:331] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c: stopping tablet replica
I20250112 20:59:00.193882 15454 raft_consensus.cc:2238] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:00.194217 15454 raft_consensus.cc:2267] T a5ee370770b54ae593e07378574861d1 P c60c0cbe377c450297e3d536b7f7c89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.262784 15454 tablet_server.cc:195] TabletServer@127.15.23.132:0 shutdown complete.
I20250112 20:59:00.323593 15454 master.cc:537] Master@127.15.23.190:45335 shutting down...
I20250112 20:59:00.347544 15454 raft_consensus.cc:2238] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:00.348016 15454 raft_consensus.cc:2267] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:00.348276 15454 tablet_replica.cc:331] T 00000000000000000000000000000000 P d64fb54f69204f7287968b68ba6d048a: stopping tablet replica
I20250112 20:59:00.368460 15454 master.cc:559] Master@127.15.23.190:45335 shutdown complete.
[ OK ] LeaderRebalancerTest.RestartTserver (34682 ms)
[ RUN ] LeaderRebalancerTest.TestMaintenanceMode
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_leader_rebalancer-test.cc:471: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
[ SKIPPED ] LeaderRebalancerTest.TestMaintenanceMode (6 ms)
[----------] 5 tests from LeaderRebalancerTest (147708 ms total)
[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20250112 20:59:00.460070 15454 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.23.190:45463
I20250112 20:59:00.462095 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:59:00.467648 19538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:00.468783 19539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:00.469949 19541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:00.470553 15454 server_base.cc:1034] running on GCE node
I20250112 20:59:00.471604 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:59:00.471796 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:59:00.471904 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715540471894 us; error 0 us; skew 500 ppm
I20250112 20:59:00.472328 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:59:00.482700 15454 webserver.cc:458] Webserver started at http://127.15.23.190:38527/ using document root <none> and password file <none>
I20250112 20:59:00.483151 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:59:00.483295 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:59:00.483501 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:59:00.484568 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/master-0-root/instance:
uuid: "ce75ee801a7543fa95ca9d8ad03ca27f"
format_stamp: "Formatted at 2025-01-12 20:59:00 on dist-test-slave-m2nm"
I20250112 20:59:00.488938 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20250112 20:59:00.492111 19546 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.492838 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250112 20:59:00.493127 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/master-0-root
uuid: "ce75ee801a7543fa95ca9d8ad03ca27f"
format_stamp: "Formatted at 2025-01-12 20:59:00 on dist-test-slave-m2nm"
I20250112 20:59:00.493376 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:59:00.506819 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:59:00.507978 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:59:00.544106 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.190:45463
I20250112 20:59:00.544194 19597 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.190:45463 every 8 connection(s)
I20250112 20:59:00.547803 19598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:00.557490 19598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f: Bootstrap starting.
I20250112 20:59:00.561954 19598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:00.565925 19598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f: No bootstrap required, opened a new log
I20250112 20:59:00.567909 19598 raft_consensus.cc:357] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce75ee801a7543fa95ca9d8ad03ca27f" member_type: VOTER }
I20250112 20:59:00.568276 19598 raft_consensus.cc:383] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:00.568542 19598 raft_consensus.cc:738] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce75ee801a7543fa95ca9d8ad03ca27f, State: Initialized, Role: FOLLOWER
I20250112 20:59:00.569063 19598 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce75ee801a7543fa95ca9d8ad03ca27f" member_type: VOTER }
I20250112 20:59:00.569594 19598 raft_consensus.cc:397] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 20:59:00.569804 19598 raft_consensus.cc:491] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 20:59:00.570050 19598 raft_consensus.cc:3054] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:00.574210 19598 raft_consensus.cc:513] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce75ee801a7543fa95ca9d8ad03ca27f" member_type: VOTER }
I20250112 20:59:00.574687 19598 leader_election.cc:304] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce75ee801a7543fa95ca9d8ad03ca27f; no voters:
I20250112 20:59:00.575749 19598 leader_election.cc:290] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 20:59:00.576148 19601 raft_consensus.cc:2798] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:00.577486 19601 raft_consensus.cc:695] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 1 LEADER]: Becoming Leader. State: Replica: ce75ee801a7543fa95ca9d8ad03ca27f, State: Running, Role: LEADER
I20250112 20:59:00.578044 19601 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce75ee801a7543fa95ca9d8ad03ca27f" member_type: VOTER }
I20250112 20:59:00.578655 19598 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [sys.catalog]: configured and running, proceeding with master startup.
I20250112 20:59:00.582105 19603 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce75ee801a7543fa95ca9d8ad03ca27f. Latest consensus state: current_term: 1 leader_uuid: "ce75ee801a7543fa95ca9d8ad03ca27f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce75ee801a7543fa95ca9d8ad03ca27f" member_type: VOTER } }
I20250112 20:59:00.582830 19603 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [sys.catalog]: This master's current role is: LEADER
I20250112 20:59:00.582741 19602 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce75ee801a7543fa95ca9d8ad03ca27f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce75ee801a7543fa95ca9d8ad03ca27f" member_type: VOTER } }
I20250112 20:59:00.583369 19602 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [sys.catalog]: This master's current role is: LEADER
I20250112 20:59:00.589555 19608 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 20:59:00.594527 19608 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 20:59:00.595595 15454 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250112 20:59:00.602403 19608 catalog_manager.cc:1348] Generated new cluster ID: bf76181ef9464d8898e07bf5579c4285
I20250112 20:59:00.602667 19608 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 20:59:00.614635 19608 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 20:59:00.615854 19608 catalog_manager.cc:1505] Loading token signing keys...
I20250112 20:59:00.630498 19608 catalog_manager.cc:5899] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f: Generated new TSK 0
I20250112 20:59:00.631052 19608 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 20:59:00.662210 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:59:00.668519 19619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:00.669485 19620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:00.670326 19622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:00.671404 15454 server_base.cc:1034] running on GCE node
I20250112 20:59:00.672245 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:59:00.672412 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:59:00.672549 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715540672537 us; error 0 us; skew 500 ppm
I20250112 20:59:00.673045 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:59:00.675199 15454 webserver.cc:458] Webserver started at http://127.15.23.129:43341/ using document root <none> and password file <none>
I20250112 20:59:00.675622 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:59:00.675777 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:59:00.675977 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:59:00.677053 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-0-root/instance:
uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
format_stamp: "Formatted at 2025-01-12 20:59:00 on dist-test-slave-m2nm"
I20250112 20:59:00.681303 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250112 20:59:00.684430 19627 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.685122 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250112 20:59:00.685366 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-0-root
uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
format_stamp: "Formatted at 2025-01-12 20:59:00 on dist-test-slave-m2nm"
I20250112 20:59:00.685616 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:59:00.700528 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:59:00.701699 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:59:00.702991 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:59:00.705034 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:59:00.705216 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.705439 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:59:00.705597 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.741811 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.129:34089
I20250112 20:59:00.741915 19689 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.129:34089 every 8 connection(s)
I20250112 20:59:00.746201 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:59:00.753670 19694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:00.754951 19695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:00.757243 15454 server_base.cc:1034] running on GCE node
I20250112 20:59:00.757896 19690 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45463
I20250112 20:59:00.758253 19690 heartbeater.cc:463] Registering TS with master...
I20250112 20:59:00.758846 19690 heartbeater.cc:510] Master 127.15.23.190:45463 requested a full tablet report, sending...
W20250112 20:59:00.758867 19697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:00.759857 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:59:00.760059 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:59:00.760227 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715540760211 us; error 0 us; skew 500 ppm
I20250112 20:59:00.760784 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:59:00.760720 19563 ts_manager.cc:194] Registered new tserver with Master: f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089)
I20250112 20:59:00.762805 19563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45346
I20250112 20:59:00.763321 15454 webserver.cc:458] Webserver started at http://127.15.23.130:38941/ using document root <none> and password file <none>
I20250112 20:59:00.763887 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:59:00.764107 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:59:00.764344 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:59:00.765378 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-1-root/instance:
uuid: "f9d40940ddca458db0f091c07f19b740"
format_stamp: "Formatted at 2025-01-12 20:59:00 on dist-test-slave-m2nm"
I20250112 20:59:00.769546 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250112 20:59:00.772651 19702 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.773283 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250112 20:59:00.773538 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-1-root
uuid: "f9d40940ddca458db0f091c07f19b740"
format_stamp: "Formatted at 2025-01-12 20:59:00 on dist-test-slave-m2nm"
I20250112 20:59:00.773795 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:59:00.781759 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:59:00.782783 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:59:00.784035 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:59:00.786192 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:59:00.786360 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.786621 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:59:00.786777 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.823524 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.130:42611
I20250112 20:59:00.823617 19764 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.130:42611 every 8 connection(s)
I20250112 20:59:00.828001 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:59:00.836139 19768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:00.837296 19769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:00.838277 19771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:00.838769 19765 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45463
I20250112 20:59:00.839103 19765 heartbeater.cc:463] Registering TS with master...
I20250112 20:59:00.839668 19765 heartbeater.cc:510] Master 127.15.23.190:45463 requested a full tablet report, sending...
I20250112 20:59:00.839679 15454 server_base.cc:1034] running on GCE node
I20250112 20:59:00.840551 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:59:00.840822 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:59:00.841027 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715540841007 us; error 0 us; skew 500 ppm
I20250112 20:59:00.841512 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:59:00.841562 19563 ts_manager.cc:194] Registered new tserver with Master: f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:00.843161 19563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45350
I20250112 20:59:00.843992 15454 webserver.cc:458] Webserver started at http://127.15.23.131:37609/ using document root <none> and password file <none>
I20250112 20:59:00.844537 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:59:00.844749 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:59:00.845003 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:59:00.846143 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-2-root/instance:
uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
format_stamp: "Formatted at 2025-01-12 20:59:00 on dist-test-slave-m2nm"
I20250112 20:59:00.850314 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250112 20:59:00.853190 19776 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.853868 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250112 20:59:00.854125 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-2-root
uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
format_stamp: "Formatted at 2025-01-12 20:59:00 on dist-test-slave-m2nm"
I20250112 20:59:00.854378 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1736715392595610-15454-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:59:00.871922 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:59:00.872998 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:59:00.874289 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:59:00.876499 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:59:00.876688 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.876905 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:59:00.877044 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:00.917217 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.131:41093
I20250112 20:59:00.917311 19838 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.131:41093 every 8 connection(s)
I20250112 20:59:00.929262 19839 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45463
I20250112 20:59:00.929669 19839 heartbeater.cc:463] Registering TS with master...
I20250112 20:59:00.930254 19839 heartbeater.cc:510] Master 127.15.23.190:45463 requested a full tablet report, sending...
I20250112 20:59:00.931847 19563 ts_manager.cc:194] Registered new tserver with Master: b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:00.931962 15454 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.011910397s
I20250112 20:59:00.933575 19563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45364
I20250112 20:59:00.934867 15454 test_util.cc:274] Using random seed: -1729843005
I20250112 20:59:00.955085 19563 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45380:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250112 20:59:00.957672 19563 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250112 20:59:01.019124 19655 tablet_service.cc:1467] Processing CreateTablet for tablet e38073eed2194f91bbed0bba42455734 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:01.020826 19655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38073eed2194f91bbed0bba42455734. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.022861 19654 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc5e3870dde48d1a07112c7c2a295a6 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:01.024475 19654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fc5e3870dde48d1a07112c7c2a295a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.026830 19653 tablet_service.cc:1467] Processing CreateTablet for tablet 6bb774316f894a44984da5099d1df61c (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:01.028318 19653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bb774316f894a44984da5099d1df61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.049115 19652 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee909bd13c44a2c85185e4ae376cab5 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:01.050526 19652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ee909bd13c44a2c85185e4ae376cab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.053653 19651 tablet_service.cc:1467] Processing CreateTablet for tablet d69c5861f39b4aa0b8396f278fb22bc9 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:01.054998 19651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69c5861f39b4aa0b8396f278fb22bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.059203 19650 tablet_service.cc:1467] Processing CreateTablet for tablet 30a1811da1844978b18649248a5a3f20 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:01.060778 19650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a1811da1844978b18649248a5a3f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.063303 19649 tablet_service.cc:1467] Processing CreateTablet for tablet 6c58a59765104b37b0cbc48a3a6a2902 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:01.064996 19649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c58a59765104b37b0cbc48a3a6a2902. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.074312 19804 tablet_service.cc:1467] Processing CreateTablet for tablet e38073eed2194f91bbed0bba42455734 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:01.075759 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38073eed2194f91bbed0bba42455734. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.077622 19803 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc5e3870dde48d1a07112c7c2a295a6 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:01.079524 19798 tablet_service.cc:1467] Processing CreateTablet for tablet 6c58a59765104b37b0cbc48a3a6a2902 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:01.079807 19802 tablet_service.cc:1467] Processing CreateTablet for tablet 6bb774316f894a44984da5099d1df61c (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:01.082144 19801 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee909bd13c44a2c85185e4ae376cab5 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:01.083195 19648 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb6313eae3d4cbd9526bde23d688b51 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:01.083709 19799 tablet_service.cc:1467] Processing CreateTablet for tablet 30a1811da1844978b18649248a5a3f20 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:01.084314 19730 tablet_service.cc:1467] Processing CreateTablet for tablet e38073eed2194f91bbed0bba42455734 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:01.084560 19648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb6313eae3d4cbd9526bde23d688b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.086414 19723 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb6313eae3d4cbd9526bde23d688b51 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:01.086911 19729 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc5e3870dde48d1a07112c7c2a295a6 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:01.085726 19730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38073eed2194f91bbed0bba42455734. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.089025 19728 tablet_service.cc:1467] Processing CreateTablet for tablet 6bb774316f894a44984da5099d1df61c (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:01.089205 19722 tablet_service.cc:1467] Processing CreateTablet for tablet 68785fca3fe943b49359b52a5545a970 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:01.090281 19723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb6313eae3d4cbd9526bde23d688b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.091226 19727 tablet_service.cc:1467] Processing CreateTablet for tablet 1ee909bd13c44a2c85185e4ae376cab5 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:01.085829 19647 tablet_service.cc:1467] Processing CreateTablet for tablet 68785fca3fe943b49359b52a5545a970 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:01.092762 19796 tablet_service.cc:1467] Processing CreateTablet for tablet 68785fca3fe943b49359b52a5545a970 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:01.093159 19726 tablet_service.cc:1467] Processing CreateTablet for tablet d69c5861f39b4aa0b8396f278fb22bc9 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:01.093974 19647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68785fca3fe943b49359b52a5545a970. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.095069 19725 tablet_service.cc:1467] Processing CreateTablet for tablet 30a1811da1844978b18649248a5a3f20 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:01.091226 19797 tablet_service.cc:1467] Processing CreateTablet for tablet 9cb6313eae3d4cbd9526bde23d688b51 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:01.086237 19800 tablet_service.cc:1467] Processing CreateTablet for tablet d69c5861f39b4aa0b8396f278fb22bc9 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:01.098089 19724 tablet_service.cc:1467] Processing CreateTablet for tablet 6c58a59765104b37b0cbc48a3a6a2902 (DEFAULT_TABLE table=test-workload [id=de8797d2d5ef4accaa886b6402b15c15]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:01.099113 19803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fc5e3870dde48d1a07112c7c2a295a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.100836 19728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bb774316f894a44984da5099d1df61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.101476 19727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ee909bd13c44a2c85185e4ae376cab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.102015 19725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a1811da1844978b18649248a5a3f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.102095 19801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ee909bd13c44a2c85185e4ae376cab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.102653 19726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69c5861f39b4aa0b8396f278fb22bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.103252 19722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68785fca3fe943b49359b52a5545a970. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.103852 19799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30a1811da1844978b18649248a5a3f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.104903 19724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c58a59765104b37b0cbc48a3a6a2902. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.107205 19800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d69c5861f39b4aa0b8396f278fb22bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.108880 19729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fc5e3870dde48d1a07112c7c2a295a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.117206 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bb774316f894a44984da5099d1df61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.120959 19797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cb6313eae3d4cbd9526bde23d688b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.123585 19796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68785fca3fe943b49359b52a5545a970. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.128906 19798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c58a59765104b37b0cbc48a3a6a2902. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:01.224393 19854 tablet_bootstrap.cc:492] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.231789 19855 tablet_bootstrap.cc:492] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.238305 19855 tablet_bootstrap.cc:654] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.245900 19854 tablet_bootstrap.cc:654] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.250547 19856 tablet_bootstrap.cc:492] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.273773 19855 tablet_bootstrap.cc:492] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.275853 19855 ts_tablet_manager.cc:1397] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.044s user 0.020s sys 0.020s
I20250112 20:59:01.273905 19854 tablet_bootstrap.cc:492] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.276953 19854 ts_tablet_manager.cc:1397] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.053s user 0.016s sys 0.016s
I20250112 20:59:01.274549 19856 tablet_bootstrap.cc:654] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.278482 19855 raft_consensus.cc:357] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.279265 19855 raft_consensus.cc:383] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.279594 19855 raft_consensus.cc:738] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.279565 19854 raft_consensus.cc:357] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.280562 19854 raft_consensus.cc:383] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.281056 19854 raft_consensus.cc:738] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.280543 19855 consensus_queue.cc:260] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.281622 19856 tablet_bootstrap.cc:492] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.282102 19856 ts_tablet_manager.cc:1397] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.001s
I20250112 20:59:01.281752 19854 consensus_queue.cc:260] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.282810 19690 heartbeater.cc:502] Master 127.15.23.190:45463 was elected leader, sending a full tablet report...
I20250112 20:59:01.283525 19855 ts_tablet_manager.cc:1428] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20250112 20:59:01.284704 19855 tablet_bootstrap.cc:492] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.284834 19856 raft_consensus.cc:357] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.285684 19856 raft_consensus.cc:383] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.285966 19856 raft_consensus.cc:738] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.286677 19856 consensus_queue.cc:260] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.291236 19854 ts_tablet_manager.cc:1428] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20250112 20:59:01.291818 19855 tablet_bootstrap.cc:654] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.292282 19854 tablet_bootstrap.cc:492] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.293036 19839 heartbeater.cc:502] Master 127.15.23.190:45463 was elected leader, sending a full tablet report...
I20250112 20:59:01.300619 19854 tablet_bootstrap.cc:654] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.303170 19855 tablet_bootstrap.cc:492] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.303809 19855 ts_tablet_manager.cc:1397] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.009s
I20250112 20:59:01.307242 19855 raft_consensus.cc:357] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.308349 19855 raft_consensus.cc:383] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.308818 19855 raft_consensus.cc:738] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.309898 19855 consensus_queue.cc:260] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.312861 19855 ts_tablet_manager.cc:1428] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.009s user 0.003s sys 0.005s
I20250112 20:59:01.314069 19855 tablet_bootstrap.cc:492] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.322018 19855 tablet_bootstrap.cc:654] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.334738 19855 tablet_bootstrap.cc:492] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.335454 19855 ts_tablet_manager.cc:1397] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.022s user 0.017s sys 0.003s
I20250112 20:59:01.339174 19855 raft_consensus.cc:357] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.340335 19855 raft_consensus.cc:383] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.340777 19855 raft_consensus.cc:738] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.341984 19855 consensus_queue.cc:260] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.345440 19855 ts_tablet_manager.cc:1428] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.010s user 0.003s sys 0.005s
I20250112 20:59:01.346884 19855 tablet_bootstrap.cc:492] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.354830 19855 tablet_bootstrap.cc:654] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.361380 19860 raft_consensus.cc:491] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.362094 19860 raft_consensus.cc:513] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.368520 19860 leader_election.cc:290] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.376765 19765 heartbeater.cc:502] Master 127.15.23.190:45463 was elected leader, sending a full tablet report...
I20250112 20:59:01.379678 19854 tablet_bootstrap.cc:492] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.380540 19854 ts_tablet_manager.cc:1397] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.088s user 0.044s sys 0.044s
I20250112 20:59:01.382375 19856 ts_tablet_manager.cc:1428] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.100s user 0.037s sys 0.056s
I20250112 20:59:01.384428 19854 raft_consensus.cc:357] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.385645 19854 raft_consensus.cc:383] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.386090 19854 raft_consensus.cc:738] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.387210 19854 consensus_queue.cc:260] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.393827 19854 ts_tablet_manager.cc:1428] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.013s user 0.002s sys 0.006s
I20250112 20:59:01.384493 19856 tablet_bootstrap.cc:492] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.395429 19854 tablet_bootstrap.cc:492] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.401189 19856 tablet_bootstrap.cc:654] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.403173 19862 raft_consensus.cc:491] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.404114 19854 tablet_bootstrap.cc:654] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.403749 19862 raft_consensus.cc:513] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.409996 19855 tablet_bootstrap.cc:492] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.410678 19855 ts_tablet_manager.cc:1397] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.064s user 0.018s sys 0.044s
I20250112 20:59:01.412711 19862 leader_election.cc:290] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.413888 19855 raft_consensus.cc:357] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.414762 19855 raft_consensus.cc:383] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.415192 19855 raft_consensus.cc:738] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.416435 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68785fca3fe943b49359b52a5545a970" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:01.416285 19855 consensus_queue.cc:260] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
W20250112 20:59:01.418198 19628 leader_election.cc:343] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.420208 19854 tablet_bootstrap.cc:492] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.420809 19854 ts_tablet_manager.cc:1397] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.026s user 0.014s sys 0.009s
I20250112 20:59:01.423357 19856 tablet_bootstrap.cc:492] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.423905 19856 ts_tablet_manager.cc:1397] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.005s
I20250112 20:59:01.423645 19854 raft_consensus.cc:357] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.428694 19854 raft_consensus.cc:383] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.429113 19854 raft_consensus.cc:738] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.429487 19861 raft_consensus.cc:491] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.426651 19855 ts_tablet_manager.cc:1428] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20250112 20:59:01.429859 19854 consensus_queue.cc:260] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.431193 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68785fca3fe943b49359b52a5545a970" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.426753 19856 raft_consensus.cc:357] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.433252 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.433424 19855 tablet_bootstrap.cc:492] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.433519 19856 raft_consensus.cc:383] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.429946 19861 raft_consensus.cc:513] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
W20250112 20:59:01.434111 19628 leader_election.cc:343] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.434532 19628 leader_election.cc:304] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740
I20250112 20:59:01.435294 19860 raft_consensus.cc:2743] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:01.434087 19856 raft_consensus.cc:738] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.437047 19856 consensus_queue.cc:260] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.439801 19856 ts_tablet_manager.cc:1428] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
W20250112 20:59:01.440248 19840 tablet.cc:2367] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250112 20:59:01.440665 19814 raft_consensus.cc:2463] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 0.
I20250112 20:59:01.442288 19856 tablet_bootstrap.cc:492] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.443480 19703 leader_election.cc:304] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:01.450357 19861 leader_election.cc:290] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:01.451705 19856 tablet_bootstrap.cc:654] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.456147 19855 tablet_bootstrap.cc:654] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.456265 19862 raft_consensus.cc:2798] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.457278 19862 raft_consensus.cc:491] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.457617 19862 raft_consensus.cc:3054] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.464689 19874 raft_consensus.cc:491] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.465289 19874 raft_consensus.cc:513] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.468709 19874 leader_election.cc:290] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:01.469614 19854 ts_tablet_manager.cc:1428] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.048s user 0.021s sys 0.010s
I20250112 20:59:01.470523 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.470979 19855 tablet_bootstrap.cc:492] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.471623 19855 ts_tablet_manager.cc:1397] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.038s user 0.005s sys 0.009s
W20250112 20:59:01.472031 19706 leader_election.cc:343] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.473893 19862 raft_consensus.cc:513] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.475800 19854 tablet_bootstrap.cc:492] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.475385 19855 raft_consensus.cc:357] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.476399 19855 raft_consensus.cc:383] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.476817 19855 raft_consensus.cc:738] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.477843 19855 consensus_queue.cc:260] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.481379 19862 leader_election.cc:290] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.482041 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:01.482468 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:01.485512 19814 raft_consensus.cc:3054] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.485637 19862 raft_consensus.cc:491] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.485875 19854 tablet_bootstrap.cc:654] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.486289 19862 raft_consensus.cc:513] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.490795 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
W20250112 20:59:01.492000 19706 leader_election.cc:343] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:59:01.492940 19706 leader_election.cc:343] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.493520 19862 leader_election.cc:290] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.494208 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
W20250112 20:59:01.495481 19703 leader_election.cc:343] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.495810 19703 leader_election.cc:304] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:01.498768 19855 ts_tablet_manager.cc:1428] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.027s user 0.006s sys 0.002s
I20250112 20:59:01.501101 19814 raft_consensus.cc:2463] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 1.
I20250112 20:59:01.504169 19703 leader_election.cc:304] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters: f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:01.505884 19877 raft_consensus.cc:2798] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:01.507095 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:01.507575 19855 tablet_bootstrap.cc:492] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.507618 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
W20250112 20:59:01.509671 19777 leader_election.cc:343] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.512310 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.512421 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.512980 19862 raft_consensus.cc:2743] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:01.513303 19665 raft_consensus.cc:2463] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
W20250112 20:59:01.514735 19780 leader_election.cc:343] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.515269 19877 raft_consensus.cc:695] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:01.515866 19780 leader_election.cc:304] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters: f9d40940ddca458db0f091c07f19b740
I20250112 20:59:01.516168 19877 consensus_queue.cc:237] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.517244 19855 tablet_bootstrap.cc:654] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.517119 19874 raft_consensus.cc:2798] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.518056 19874 raft_consensus.cc:491] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.518441 19874 raft_consensus.cc:3054] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.520545 19856 tablet_bootstrap.cc:492] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.521123 19856 ts_tablet_manager.cc:1397] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.079s user 0.039s sys 0.010s
I20250112 20:59:01.522313 19854 tablet_bootstrap.cc:492] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.522990 19854 ts_tablet_manager.cc:1397] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.047s user 0.010s sys 0.003s
I20250112 20:59:01.523988 19856 raft_consensus.cc:357] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.524845 19856 raft_consensus.cc:383] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.525231 19856 raft_consensus.cc:738] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.525377 19874 raft_consensus.cc:513] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.526108 19856 consensus_queue.cc:260] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.529742 19777 leader_election.cc:304] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9; no voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740
I20250112 20:59:01.530820 19861 raft_consensus.cc:2743] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:01.531591 19856 ts_tablet_manager.cc:1428] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.010s user 0.004s sys 0.002s
I20250112 20:59:01.526437 19854 raft_consensus.cc:357] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.532810 19854 raft_consensus.cc:383] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.532853 19856 tablet_bootstrap.cc:492] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.533166 19854 raft_consensus.cc:738] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.532492 19563 catalog_manager.cc:5526] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:01.533622 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:01.534180 19665 raft_consensus.cc:3054] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.533995 19854 consensus_queue.cc:260] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.537233 19854 ts_tablet_manager.cc:1428] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20250112 20:59:01.539920 19856 tablet_bootstrap.cc:654] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.544622 19874 leader_election.cc:290] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:01.545764 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:01.546090 19665 raft_consensus.cc:2463] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 1.
I20250112 20:59:01.547430 19780 leader_election.cc:304] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
W20250112 20:59:01.548136 19777 leader_election.cc:343] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.548244 19854 tablet_bootstrap.cc:492] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.549190 19874 raft_consensus.cc:2798] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:01.549940 19874 raft_consensus.cc:695] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:01.550025 19855 tablet_bootstrap.cc:492] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.550745 19855 ts_tablet_manager.cc:1397] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.043s user 0.013s sys 0.004s
I20250112 20:59:01.550909 19874 consensus_queue.cc:237] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.553586 19855 raft_consensus.cc:357] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.554435 19855 raft_consensus.cc:383] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.554842 19855 raft_consensus.cc:738] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.555335 19856 tablet_bootstrap.cc:492] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.555915 19856 ts_tablet_manager.cc:1397] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.005s
I20250112 20:59:01.555737 19855 consensus_queue.cc:260] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.557066 19854 tablet_bootstrap.cc:654] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.559073 19855 ts_tablet_manager.cc:1428] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250112 20:59:01.562713 19855 tablet_bootstrap.cc:492] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.559434 19856 raft_consensus.cc:357] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.566474 19856 raft_consensus.cc:383] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.566778 19856 raft_consensus.cc:738] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.567485 19856 consensus_queue.cc:260] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.569871 19856 ts_tablet_manager.cc:1428] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20250112 20:59:01.570674 19854 tablet_bootstrap.cc:492] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.571331 19855 tablet_bootstrap.cc:654] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.571333 19854 ts_tablet_manager.cc:1397] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.006s
I20250112 20:59:01.573230 19856 tablet_bootstrap.cc:492] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.574995 19854 raft_consensus.cc:357] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.575827 19854 raft_consensus.cc:383] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.576150 19854 raft_consensus.cc:738] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.576956 19854 consensus_queue.cc:260] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.579489 19854 ts_tablet_manager.cc:1428] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20250112 20:59:01.580958 19856 tablet_bootstrap.cc:654] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.583372 19854 tablet_bootstrap.cc:492] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.584569 19855 tablet_bootstrap.cc:492] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.585340 19855 ts_tablet_manager.cc:1397] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.002s
I20250112 20:59:01.590215 19854 tablet_bootstrap.cc:654] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.590384 19855 raft_consensus.cc:357] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
W20250112 20:59:01.591113 19611 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:59:01.591780 19855 raft_consensus.cc:383] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.592250 19855 raft_consensus.cc:738] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.592221 19563 catalog_manager.cc:5526] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:01.593806 19855 consensus_queue.cc:260] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.596096 19856 tablet_bootstrap.cc:492] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.596665 19856 ts_tablet_manager.cc:1397] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.008s
I20250112 20:59:01.596804 19855 ts_tablet_manager.cc:1428] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.011s user 0.006s sys 0.002s
I20250112 20:59:01.597795 19855 tablet_bootstrap.cc:492] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.599363 19856 raft_consensus.cc:357] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.600082 19856 raft_consensus.cc:383] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.600353 19856 raft_consensus.cc:738] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.601305 19856 consensus_queue.cc:260] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.603644 19855 tablet_bootstrap.cc:654] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.603971 19856 ts_tablet_manager.cc:1428] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:59:01.604975 19856 tablet_bootstrap.cc:492] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.610909 19860 raft_consensus.cc:491] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.611428 19860 raft_consensus.cc:513] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.612041 19856 tablet_bootstrap.cc:654] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.612846 19877 raft_consensus.cc:491] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.613458 19860 leader_election.cc:290] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.613327 19877 raft_consensus.cc:513] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.615298 19877 leader_election.cc:290] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.615981 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.618786 19814 raft_consensus.cc:2388] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f0b8ad29373745ddac1bcb23ca06cf25 in current term 1: Already voted for candidate f9d40940ddca458db0f091c07f19b740 in this term.
I20250112 20:59:01.617118 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.617298 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
W20250112 20:59:01.620218 19706 leader_election.cc:343] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Illegal state: must be running to vote when last-logged opid is not known
W20250112 20:59:01.620954 19703 leader_election.cc:343] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.621345 19703 leader_election.cc:304] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:01.616279 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:01.617997 19854 tablet_bootstrap.cc:492] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.623189 19854 ts_tablet_manager.cc:1397] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.040s user 0.022s sys 0.007s
I20250112 20:59:01.623312 19860 raft_consensus.cc:491] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.623965 19860 raft_consensus.cc:513] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.624370 19628 leader_election.cc:304] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740
I20250112 20:59:01.626431 19860 leader_election.cc:290] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.626955 19854 raft_consensus.cc:357] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.627852 19877 raft_consensus.cc:2743] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:01.632460 19854 raft_consensus.cc:383] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.632774 19856 tablet_bootstrap.cc:492] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.632477 19874 raft_consensus.cc:491] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.633625 19856 ts_tablet_manager.cc:1397] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.029s user 0.016s sys 0.006s
I20250112 20:59:01.633705 19874 raft_consensus.cc:513] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.635772 19874 leader_election.cc:290] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:01.636840 19855 tablet_bootstrap.cc:492] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.636651 19856 raft_consensus.cc:357] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.637054 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.636966 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:01.637416 19856 raft_consensus.cc:383] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.632841 19854 raft_consensus.cc:738] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.637794 19856 raft_consensus.cc:738] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.637809 19665 raft_consensus.cc:2463] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
W20250112 20:59:01.638545 19777 leader_election.cc:343] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.638370 19854 consensus_queue.cc:260] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.637300 19855 ts_tablet_manager.cc:1397] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.005s
I20250112 20:59:01.638806 19856 consensus_queue.cc:260] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.639221 19780 leader_election.cc:304] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters: f9d40940ddca458db0f091c07f19b740
I20250112 20:59:01.640650 19861 raft_consensus.cc:2798] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.640978 19854 ts_tablet_manager.cc:1428] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.017s user 0.004s sys 0.003s
I20250112 20:59:01.640980 19861 raft_consensus.cc:491] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.641789 19856 ts_tablet_manager.cc:1428] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20250112 20:59:01.642000 19854 tablet_bootstrap.cc:492] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.641703 19855 raft_consensus.cc:357] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.642630 19855 raft_consensus.cc:383] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.642932 19855 raft_consensus.cc:738] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.643047 19856 tablet_bootstrap.cc:492] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.643688 19855 consensus_queue.cc:260] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.633735 19860 raft_consensus.cc:3054] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.645833 19855 ts_tablet_manager.cc:1428] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:59:01.641866 19861 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.649883 19856 tablet_bootstrap.cc:654] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.652112 19854 tablet_bootstrap.cc:654] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.653605 19855 tablet_bootstrap.cc:492] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:01.659623 19855 tablet_bootstrap.cc:654] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.661270 19856 tablet_bootstrap.cc:492] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.661813 19856 ts_tablet_manager.cc:1397] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.003s
I20250112 20:59:01.662178 19861 raft_consensus.cc:513] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.659648 19860 raft_consensus.cc:2743] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:01.664965 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:01.666800 19665 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.667989 19877 raft_consensus.cc:491] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250112 20:59:01.667846 19628 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.130:42611, user_credentials={real_user=slave}} blocked reactor thread for 43550.6us
I20250112 20:59:01.668622 19877 raft_consensus.cc:513] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.669310 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:01.670055 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:01.670637 19877 leader_election.cc:290] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.670831 19739 raft_consensus.cc:2463] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
W20250112 20:59:01.670655 19777 leader_election.cc:343] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.672307 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.672865 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.673250 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.673084 19664 raft_consensus.cc:2388] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9d40940ddca458db0f091c07f19b740 in current term 1: Already voted for candidate b24e8e78a6834a6fae06c193b5ac43a9 in this term.
I20250112 20:59:01.674089 19861 leader_election.cc:290] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:01.674278 19628 leader_election.cc:304] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
W20250112 20:59:01.675217 19628 leader_election.cc:343] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.675478 19703 leader_election.cc:304] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:01.676632 19877 raft_consensus.cc:3054] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.678750 19665 raft_consensus.cc:2463] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 1.
I20250112 20:59:01.679141 19860 raft_consensus.cc:2798] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.680003 19780 leader_election.cc:304] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters: f9d40940ddca458db0f091c07f19b740
I20250112 20:59:01.680457 19860 raft_consensus.cc:491] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.680873 19860 raft_consensus.cc:3054] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.681275 19861 raft_consensus.cc:2798] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:01.683337 19877 raft_consensus.cc:2743] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:01.664674 19856 raft_consensus.cc:357] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.683990 19856 raft_consensus.cc:383] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.684013 19882 raft_consensus.cc:491] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.684475 19856 raft_consensus.cc:738] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.684839 19882 raft_consensus.cc:513] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.685251 19856 consensus_queue.cc:260] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.683045 19854 tablet_bootstrap.cc:492] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.686966 19854 ts_tablet_manager.cc:1397] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.045s user 0.010s sys 0.001s
I20250112 20:59:01.686933 19882 leader_election.cc:290] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.689128 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.689790 19814 raft_consensus.cc:2463] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 0.
I20250112 20:59:01.689911 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.690722 19665 raft_consensus.cc:2463] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 0.
I20250112 20:59:01.691171 19856 ts_tablet_manager.cc:1428] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20250112 20:59:01.690984 19703 leader_election.cc:304] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:01.692144 19856 tablet_bootstrap.cc:492] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.692205 19882 raft_consensus.cc:2798] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.692808 19882 raft_consensus.cc:491] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.693169 19882 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.694175 19861 raft_consensus.cc:695] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:01.695083 19861 consensus_queue.cc:237] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.698390 19856 tablet_bootstrap.cc:654] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.689913 19854 raft_consensus.cc:357] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.699285 19854 raft_consensus.cc:383] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.699592 19854 raft_consensus.cc:738] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.700423 19854 consensus_queue.cc:260] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.702517 19855 tablet_bootstrap.cc:492] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:01.702906 19855 ts_tablet_manager.cc:1397] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.012s
I20250112 20:59:01.704629 19855 raft_consensus.cc:357] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.705271 19855 raft_consensus.cc:383] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.705523 19855 raft_consensus.cc:738] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.706125 19855 consensus_queue.cc:260] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.706583 19854 ts_tablet_manager.cc:1428] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20250112 20:59:01.708014 19854 tablet_bootstrap.cc:492] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.707437 19562 catalog_manager.cc:5526] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:01.709282 19860 raft_consensus.cc:513] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.700191 19882 raft_consensus.cc:513] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.712126 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:01.712913 19739 raft_consensus.cc:3054] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.713032 19860 leader_election.cc:290] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.713553 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:01.714093 19814 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.714816 19855 ts_tablet_manager.cc:1428] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20250112 20:59:01.716615 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
W20250112 20:59:01.717731 19628 leader_election.cc:343] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:01.721643 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:01.721868 19814 raft_consensus.cc:2463] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 1.
I20250112 20:59:01.722239 19665 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.722904 19703 leader_election.cc:304] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:01.723719 19877 raft_consensus.cc:2798] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:01.724146 19877 raft_consensus.cc:695] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:01.724932 19877 consensus_queue.cc:237] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.727058 19739 raft_consensus.cc:2463] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:01.728240 19628 leader_election.cc:304] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:01.728837 19665 raft_consensus.cc:2463] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 1.
I20250112 20:59:01.729277 19896 raft_consensus.cc:2798] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:01.729936 19896 raft_consensus.cc:695] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:01.730921 19896 consensus_queue.cc:237] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.735982 19882 leader_election.cc:290] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.741250 19856 tablet_bootstrap.cc:492] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.741917 19856 ts_tablet_manager.cc:1397] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.050s user 0.019s sys 0.013s
I20250112 20:59:01.742180 19854 tablet_bootstrap.cc:654] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.743257 19559 catalog_manager.cc:5526] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:01.744354 19856 raft_consensus.cc:357] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.745118 19856 raft_consensus.cc:383] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.745416 19856 raft_consensus.cc:738] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.746212 19856 consensus_queue.cc:260] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.746984 19854 tablet_bootstrap.cc:492] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.747481 19854 ts_tablet_manager.cc:1397] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.040s user 0.008s sys 0.004s
I20250112 20:59:01.747174 19562 catalog_manager.cc:5526] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:01.748725 19856 ts_tablet_manager.cc:1428] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250112 20:59:01.749643 19856 tablet_bootstrap.cc:492] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:01.749936 19854 raft_consensus.cc:357] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.750605 19854 raft_consensus.cc:383] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.750867 19854 raft_consensus.cc:738] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.751643 19854 consensus_queue.cc:260] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.754506 19854 ts_tablet_manager.cc:1428] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250112 20:59:01.755131 19856 tablet_bootstrap.cc:654] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.755300 19854 tablet_bootstrap.cc:492] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:01.759960 19856 tablet_bootstrap.cc:492] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:01.759954 19854 tablet_bootstrap.cc:654] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:01.760692 19856 ts_tablet_manager.cc:1397] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20250112 20:59:01.762825 19856 raft_consensus.cc:357] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.763485 19856 raft_consensus.cc:383] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.763777 19856 raft_consensus.cc:738] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.765405 19854 tablet_bootstrap.cc:492] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:01.765206 19856 consensus_queue.cc:260] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.765913 19854 ts_tablet_manager.cc:1397] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20250112 20:59:01.767633 19856 ts_tablet_manager.cc:1428] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250112 20:59:01.768397 19854 raft_consensus.cc:357] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.769115 19854 raft_consensus.cc:383] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:01.769374 19854 raft_consensus.cc:738] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:01.770025 19854 consensus_queue.cc:260] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.771908 19854 ts_tablet_manager.cc:1428] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:59:01.827935 19882 raft_consensus.cc:491] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.828477 19882 raft_consensus.cc:513] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.828881 19896 raft_consensus.cc:491] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.829360 19896 raft_consensus.cc:513] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.830421 19882 leader_election.cc:290] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.830969 19896 leader_election.cc:290] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.831264 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.832041 19665 raft_consensus.cc:2388] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f9d40940ddca458db0f091c07f19b740 in current term 1: Already voted for candidate b24e8e78a6834a6fae06c193b5ac43a9 in this term.
I20250112 20:59:01.831971 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.832792 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a1811da1844978b18649248a5a3f20" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.833253 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a1811da1844978b18649248a5a3f20" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:01.833417 19813 raft_consensus.cc:2463] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:01.833480 19703 leader_election.cc:304] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:01.833884 19739 raft_consensus.cc:2463] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:01.834395 19882 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.834348 19628 leader_election.cc:304] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:01.835201 19896 raft_consensus.cc:2798] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.835557 19896 raft_consensus.cc:491] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.835839 19896 raft_consensus.cc:3054] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.839121 19882 raft_consensus.cc:2743] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:01.840157 19896 raft_consensus.cc:513] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.841724 19896 leader_election.cc:290] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.842356 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a1811da1844978b18649248a5a3f20" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:01.842592 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a1811da1844978b18649248a5a3f20" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:01.842810 19739 raft_consensus.cc:3054] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.843083 19813 raft_consensus.cc:3054] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.847175 19739 raft_consensus.cc:2463] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:01.847465 19813 raft_consensus.cc:2463] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:01.848263 19628 leader_election.cc:304] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:01.848920 19896 raft_consensus.cc:2798] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:01.849390 19896 raft_consensus.cc:695] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:01.849983 19896 consensus_queue.cc:237] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.856778 19559 catalog_manager.cc:5526] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:01.857757 19882 raft_consensus.cc:491] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.858119 19882 raft_consensus.cc:513] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.859477 19882 leader_election.cc:290] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.860248 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68785fca3fe943b49359b52a5545a970" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.860363 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68785fca3fe943b49359b52a5545a970" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.861048 19813 raft_consensus.cc:2463] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 0.
I20250112 20:59:01.861007 19665 raft_consensus.cc:2463] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 0.
I20250112 20:59:01.862012 19703 leader_election.cc:304] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:01.862706 19882 raft_consensus.cc:2798] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.862987 19882 raft_consensus.cc:491] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.863351 19882 raft_consensus.cc:3054] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.867727 19882 raft_consensus.cc:513] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.869096 19882 leader_election.cc:290] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.869771 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68785fca3fe943b49359b52a5545a970" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:01.869944 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68785fca3fe943b49359b52a5545a970" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:01.870230 19665 raft_consensus.cc:3054] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.870426 19813 raft_consensus.cc:3054] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.874559 19665 raft_consensus.cc:2463] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 1.
I20250112 20:59:01.874738 19813 raft_consensus.cc:2463] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 1.
I20250112 20:59:01.875420 19706 leader_election.cc:304] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:01.876165 19882 raft_consensus.cc:2798] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:01.876521 19882 raft_consensus.cc:695] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:01.877125 19882 consensus_queue.cc:237] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.883571 19559 catalog_manager.cc:5526] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:01.916704 19882 consensus_queue.cc:1035] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:01.927371 19882 consensus_queue.cc:1035] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:01.927906 19861 raft_consensus.cc:491] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.928747 19861 raft_consensus.cc:513] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.930717 19861 leader_election.cc:290] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:01.931615 19860 raft_consensus.cc:491] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:01.932126 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:01.932958 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:01.932117 19860 raft_consensus.cc:513] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.933709 19739 raft_consensus.cc:2463] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
I20250112 20:59:01.935325 19860 leader_election.cc:290] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.935629 19777 leader_election.cc:304] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:01.936622 19664 raft_consensus.cc:2463] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
I20250112 20:59:01.937427 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:01.938277 19813 raft_consensus.cc:2463] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:01.938835 19861 raft_consensus.cc:2798] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.939177 19861 raft_consensus.cc:491] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.939445 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:01.940073 19739 raft_consensus.cc:2463] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:01.939481 19861 raft_consensus.cc:3054] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.940490 19628 leader_election.cc:304] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:01.941270 19860 raft_consensus.cc:2798] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:01.941669 19860 raft_consensus.cc:491] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:01.941931 19860 raft_consensus.cc:3054] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.947355 19861 raft_consensus.cc:513] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.947916 19860 raft_consensus.cc:513] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.949199 19861 leader_election.cc:290] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:01.951036 19860 leader_election.cc:290] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:01.951408 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:01.951915 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:01.951987 19739 raft_consensus.cc:3054] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:01.953411 19861 consensus_queue.cc:1035] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:01.953590 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:01.954044 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:01.955128 19813 raft_consensus.cc:2388] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f0b8ad29373745ddac1bcb23ca06cf25 in current term 1: Already voted for candidate b24e8e78a6834a6fae06c193b5ac43a9 in this term.
I20250112 20:59:01.955477 19665 raft_consensus.cc:2388] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b24e8e78a6834a6fae06c193b5ac43a9 in current term 1: Already voted for candidate f0b8ad29373745ddac1bcb23ca06cf25 in this term.
I20250112 20:59:01.956172 19628 leader_election.cc:304] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740
I20250112 20:59:01.959786 19860 raft_consensus.cc:2743] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250112 20:59:01.960920 19739 raft_consensus.cc:2463] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 1.
I20250112 20:59:01.961978 19777 leader_election.cc:304] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters: f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:01.962559 19861 raft_consensus.cc:2798] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:01.963208 19861 raft_consensus.cc:695] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:01.965560 19861 consensus_queue.cc:237] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:01.974264 19861 consensus_queue.cc:1035] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:01.977782 19560 catalog_manager.cc:5526] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:02.047981 19860 raft_consensus.cc:491] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:02.048467 19860 raft_consensus.cc:513] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.050062 19860 leader_election.cc:290] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.050722 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:02.051064 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:02.051431 19740 raft_consensus.cc:2463] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:02.051693 19813 raft_consensus.cc:2463] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:02.052384 19628 leader_election.cc:304] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:02.052994 19860 raft_consensus.cc:2798] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:02.053335 19860 raft_consensus.cc:491] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:02.053617 19860 raft_consensus.cc:3054] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:02.055392 19874 raft_consensus.cc:491] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:02.055843 19874 raft_consensus.cc:513] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.057819 19874 leader_election.cc:290] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:02.058251 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:02.058436 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:02.058573 19860 raft_consensus.cc:513] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.059141 19740 raft_consensus.cc:2388] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b24e8e78a6834a6fae06c193b5ac43a9 in current term 1: Already voted for candidate f0b8ad29373745ddac1bcb23ca06cf25 in this term.
I20250112 20:59:02.060101 19777 leader_election.cc:304] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9; no voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740
I20250112 20:59:02.060490 19860 leader_election.cc:290] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.060709 19874 raft_consensus.cc:3054] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:02.061233 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:02.061849 19740 raft_consensus.cc:3054] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:02.062297 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.062755 19813 raft_consensus.cc:3054] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:02.067023 19813 raft_consensus.cc:2463] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:02.067265 19874 raft_consensus.cc:2743] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:02.067811 19628 leader_election.cc:304] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:02.068290 19740 raft_consensus.cc:2463] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:02.068439 19860 raft_consensus.cc:2798] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:02.068917 19860 raft_consensus.cc:695] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:02.069594 19860 consensus_queue.cc:237] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.076721 19562 catalog_manager.cc:5526] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.089022 19874 consensus_queue.cc:1035] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.098695 19917 consensus_queue.cc:1035] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.173789 19860 consensus_queue.cc:1035] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.178097 19739 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e38073eed2194f91bbed0bba42455734"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
mode: GRACEFUL
new_leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:46774
I20250112 20:59:02.178817 19739 raft_consensus.cc:604] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Received request to transfer leadership to TS f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:02.185247 19896 consensus_queue.cc:1035] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.186643 19740 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
mode: GRACEFUL
new_leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:46774
I20250112 20:59:02.187340 19740 raft_consensus.cc:604] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Received request to transfer leadership to TS f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:02.203545 19813 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6bb774316f894a44984da5099d1df61c"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
mode: GRACEFUL
new_leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:57364
I20250112 20:59:02.204012 19813 raft_consensus.cc:604] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Received request to transfer leadership to TS f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:02.208205 19813 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
mode: GRACEFUL
new_leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:57364
I20250112 20:59:02.208693 19813 raft_consensus.cc:604] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Received request to transfer leadership to TS f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:02.212637 19813 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
mode: GRACEFUL
new_leader_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:57364
I20250112 20:59:02.213078 19813 raft_consensus.cc:604] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Received request to transfer leadership to TS f9d40940ddca458db0f091c07f19b740
I20250112 20:59:02.224943 19665 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "30a1811da1844978b18649248a5a3f20"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
mode: GRACEFUL
new_leader_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:32898
I20250112 20:59:02.225438 19665 raft_consensus.cc:604] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Received request to transfer leadership to TS f9d40940ddca458db0f091c07f19b740
I20250112 20:59:02.229561 19665 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
mode: GRACEFUL
new_leader_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:32898
I20250112 20:59:02.229984 19665 raft_consensus.cc:604] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Received request to transfer leadership to TS f9d40940ddca458db0f091c07f19b740
I20250112 20:59:02.233938 19665 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
mode: GRACEFUL
new_leader_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:32898
I20250112 20:59:02.234346 19665 raft_consensus.cc:604] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Received request to transfer leadership to TS f9d40940ddca458db0f091c07f19b740
I20250112 20:59:02.238723 19882 consensus_queue.cc:1035] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.239200 19740 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "68785fca3fe943b49359b52a5545a970"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
mode: GRACEFUL
new_leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:46774
I20250112 20:59:02.239840 19740 raft_consensus.cc:604] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Received request to transfer leadership to TS b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:02.240223 19882 consensus_queue.cc:1035] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.249845 19862 raft_consensus.cc:988] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740: : Instructing follower f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:02.250280 19862 raft_consensus.cc:1076] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Signalling peer f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:02.252458 19664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:32882
I20250112 20:59:02.252980 19664 raft_consensus.cc:491] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.253206 19664 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.260710 19664 raft_consensus.cc:513] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.261432 19665 raft_consensus.cc:1235] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Rejecting Update request from peer f9d40940ddca458db0f091c07f19b740 for earlier term 1. Current term is 2. Ops: []
I20250112 20:59:02.262099 19664 leader_election.cc:290] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.262879 19877 consensus_queue.cc:1046] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.263084 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.263378 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:02.263722 19862 raft_consensus.cc:3049] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.263718 19813 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.264278 19862 raft_consensus.cc:738] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:02.264922 19862 consensus_queue.cc:260] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.265825 19862 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.269071 19813 raft_consensus.cc:2463] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:02.270015 19628 leader_election.cc:304] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:02.270700 19896 raft_consensus.cc:2798] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.271104 19896 raft_consensus.cc:695] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:02.271816 19896 consensus_queue.cc:237] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.275413 19740 raft_consensus.cc:2463] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:02.278605 19559 catalog_manager.cc:5526] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 1 to 2, leader changed from f9d40940ddca458db0f091c07f19b740 (127.15.23.130) to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.343286 19862 consensus_queue.cc:1035] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.353358 19877 consensus_queue.cc:1035] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.358131 19896 consensus_queue.cc:1035] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.361235 19877 raft_consensus.cc:988] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740: : Instructing follower b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:02.361675 19862 raft_consensus.cc:1076] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Signalling peer b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:02.363310 19813 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "68785fca3fe943b49359b52a5545a970"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:57350
I20250112 20:59:02.363875 19813 raft_consensus.cc:491] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.364197 19813 raft_consensus.cc:3054] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.365880 19926 raft_consensus.cc:988] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25: : Instructing follower f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:02.366335 19926 raft_consensus.cc:1076] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Signalling peer f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:02.367782 19740 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "30a1811da1844978b18649248a5a3f20"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:46744
I20250112 20:59:02.368304 19740 raft_consensus.cc:491] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.368754 19740 raft_consensus.cc:3054] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.370767 19860 consensus_queue.cc:1035] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.375769 19740 raft_consensus.cc:513] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.376567 19813 raft_consensus.cc:513] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.377086 19739 raft_consensus.cc:1235] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Rejecting Update request from peer f0b8ad29373745ddac1bcb23ca06cf25 for earlier term 1. Current term is 2. Ops: []
I20250112 20:59:02.378121 19740 leader_election.cc:290] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.379145 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a1811da1844978b18649248a5a3f20" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.379436 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68785fca3fe943b49359b52a5545a970" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:02.380096 19664 raft_consensus.cc:3054] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.382926 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30a1811da1844978b18649248a5a3f20" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:02.383541 19665 raft_consensus.cc:3049] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.383845 19665 raft_consensus.cc:738] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:02.384608 19665 consensus_queue.cc:260] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.385473 19860 consensus_queue.cc:1046] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.386538 19665 raft_consensus.cc:3054] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.387039 19664 raft_consensus.cc:2463] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 2.
I20250112 20:59:02.388053 19780 leader_election.cc:304] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:02.388204 19813 leader_election.cc:290] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:02.389321 19917 raft_consensus.cc:2798] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.389737 19917 raft_consensus.cc:695] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:02.390439 19917 consensus_queue.cc:237] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.392508 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68785fca3fe943b49359b52a5545a970" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:02.393211 19739 raft_consensus.cc:3049] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.393290 19665 raft_consensus.cc:2463] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:02.393584 19739 raft_consensus.cc:738] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:02.394344 19706 leader_election.cc:304] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:02.394428 19739 consensus_queue.cc:260] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.395229 19862 raft_consensus.cc:2798] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.395689 19862 raft_consensus.cc:695] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:02.395807 19739 raft_consensus.cc:3054] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.396507 19862 consensus_queue.cc:237] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.398276 19559 catalog_manager.cc:5526] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 1 to 2, leader changed from f9d40940ddca458db0f091c07f19b740 (127.15.23.130) to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 2 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:02.402290 19739 raft_consensus.cc:2463] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 2.
I20250112 20:59:02.405853 19560 catalog_manager.cc:5526] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 1 to 2, leader changed from f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129) to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.430115 19917 consensus_queue.cc:1035] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.439266 19918 consensus_queue.cc:1035] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.446321 19918 raft_consensus.cc:988] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9: : Instructing follower f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:02.446676 19918 raft_consensus.cc:1076] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Signalling peer f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:02.448014 19664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6bb774316f894a44984da5099d1df61c"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:32896
I20250112 20:59:02.448544 19664 raft_consensus.cc:491] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.448848 19664 raft_consensus.cc:3054] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.451231 19862 raft_consensus.cc:988] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740: : Instructing follower f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:02.451679 19862 raft_consensus.cc:1076] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Signalling peer f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:02.454222 19664 raft_consensus.cc:513] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.454543 19665 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e38073eed2194f91bbed0bba42455734"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:32882
I20250112 20:59:02.455127 19665 raft_consensus.cc:491] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.455526 19665 raft_consensus.cc:3054] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.456805 19664 leader_election.cc:290] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.457801 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:02.458034 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bb774316f894a44984da5099d1df61c" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.458401 19739 raft_consensus.cc:3054] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.458624 19813 raft_consensus.cc:3049] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.458990 19813 raft_consensus.cc:738] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:02.459713 19813 consensus_queue.cc:260] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.461331 19813 raft_consensus.cc:3054] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.461710 19665 raft_consensus.cc:513] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.464275 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:02.464912 19740 raft_consensus.cc:3049] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.464342 19665 leader_election.cc:290] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.465291 19740 raft_consensus.cc:738] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:02.466102 19740 consensus_queue.cc:260] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.464977 19739 raft_consensus.cc:2463] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:02.467056 19813 raft_consensus.cc:2463] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:02.467363 19740 raft_consensus.cc:3054] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.467861 19628 leader_election.cc:304] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:02.468153 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e38073eed2194f91bbed0bba42455734" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.468732 19860 raft_consensus.cc:2798] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.468791 19812 raft_consensus.cc:3054] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.469273 19860 raft_consensus.cc:695] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:02.470104 19860 consensus_queue.cc:237] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.473879 19812 raft_consensus.cc:2463] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:02.474752 19628 leader_election.cc:304] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:02.475252 19740 raft_consensus.cc:2463] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:02.475356 19860 raft_consensus.cc:2798] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.476073 19860 raft_consensus.cc:695] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:02.476789 19860 consensus_queue.cc:237] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.477504 19562 catalog_manager.cc:5526] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 1 to 2, leader changed from b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131) to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.488086 19562 catalog_manager.cc:5526] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 1 to 2, leader changed from f9d40940ddca458db0f091c07f19b740 (127.15.23.130) to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.495942 19860 consensus_queue.cc:1035] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.500914 19926 raft_consensus.cc:988] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25: : Instructing follower f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:02.501348 19896 raft_consensus.cc:1076] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Signalling peer f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:02.503003 19739 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:46744
I20250112 20:59:02.504153 19739 raft_consensus.cc:491] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.504573 19739 raft_consensus.cc:3054] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.506134 19860 consensus_queue.cc:1035] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:02.514997 19739 raft_consensus.cc:513] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.517313 19739 leader_election.cc:290] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.517889 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.518543 19812 raft_consensus.cc:3054] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.519446 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cb6313eae3d4cbd9526bde23d688b51" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:02.520095 19665 raft_consensus.cc:3049] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.520421 19665 raft_consensus.cc:738] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:02.521178 19665 consensus_queue.cc:260] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.522471 19665 raft_consensus.cc:3054] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.525058 19812 raft_consensus.cc:2463] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:02.526137 19703 leader_election.cc:304] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:02.526759 19882 raft_consensus.cc:2798] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.527206 19882 raft_consensus.cc:695] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:02.527990 19882 consensus_queue.cc:237] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.528481 19665 raft_consensus.cc:2463] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:02.534305 19559 catalog_manager.cc:5526] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 1 to 2, leader changed from f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129) to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.554844 19861 raft_consensus.cc:988] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9: : Instructing follower f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:02.555310 19917 raft_consensus.cc:1076] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Signalling peer f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:02.556713 19665 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:32896
I20250112 20:59:02.557155 19665 raft_consensus.cc:491] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.557425 19665 raft_consensus.cc:3054] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.561899 19665 raft_consensus.cc:513] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.563341 19665 leader_election.cc:290] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.563987 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:02.564271 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ee909bd13c44a2c85185e4ae376cab5" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.564610 19739 raft_consensus.cc:3054] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.564841 19812 raft_consensus.cc:3049] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.565163 19812 raft_consensus.cc:738] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:02.565761 19812 consensus_queue.cc:260] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.566643 19812 raft_consensus.cc:3054] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.569095 19739 raft_consensus.cc:2463] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:02.569931 19628 leader_election.cc:304] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:02.570523 19860 raft_consensus.cc:2798] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.570976 19860 raft_consensus.cc:695] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:02.571372 19812 raft_consensus.cc:2463] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:02.571800 19860 consensus_queue.cc:237] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.577697 19559 catalog_manager.cc:5526] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 1 to 2, leader changed from b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131) to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.619508 19861 raft_consensus.cc:988] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9: : Instructing follower f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:02.619974 19917 raft_consensus.cc:1076] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Signalling peer f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:02.621338 19739 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:46760
I20250112 20:59:02.621747 19739 raft_consensus.cc:491] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.621989 19739 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.626422 19739 raft_consensus.cc:513] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.627737 19739 leader_election.cc:290] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.628384 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:02.628636 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.628978 19665 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.629207 19812 raft_consensus.cc:3049] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.629505 19812 raft_consensus.cc:738] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:02.630075 19812 consensus_queue.cc:260] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.630915 19812 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.633317 19665 raft_consensus.cc:2463] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:02.634462 19706 leader_election.cc:304] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:02.635226 19882 raft_consensus.cc:2798] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.635483 19812 raft_consensus.cc:2463] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:02.635603 19882 raft_consensus.cc:695] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:02.636518 19882 consensus_queue.cc:237] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.642067 19559 catalog_manager.cc:5526] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 1 to 2, leader changed from b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131) to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.724462 19896 raft_consensus.cc:988] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25: : Instructing follower f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:02.725062 19860 raft_consensus.cc:1076] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Signalling peer f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:02.726243 19739 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:46744
I20250112 20:59:02.726655 19739 raft_consensus.cc:491] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:02.726895 19739 raft_consensus.cc:3054] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.731235 19739 raft_consensus.cc:513] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.732575 19739 leader_election.cc:290] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:02.733263 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:02.733520 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c58a59765104b37b0cbc48a3a6a2902" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:02.733764 19665 raft_consensus.cc:3049] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:02.734025 19812 raft_consensus.cc:3054] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.734036 19665 raft_consensus.cc:738] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:02.734759 19665 consensus_queue.cc:260] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.735610 19665 raft_consensus.cc:3054] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.738363 19812 raft_consensus.cc:2463] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:02.739192 19703 leader_election.cc:304] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:02.739889 19882 raft_consensus.cc:2798] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:02.740023 19665 raft_consensus.cc:2463] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:02.740226 19882 raft_consensus.cc:695] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:02.741063 19882 consensus_queue.cc:237] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:02.746593 19559 catalog_manager.cc:5526] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 1 to 2, leader changed from f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129) to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:02.778071 19739 raft_consensus.cc:1270] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.779129 19860 consensus_queue.cc:1035] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.782734 19812 raft_consensus.cc:3054] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:02.786713 19813 raft_consensus.cc:1270] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.788437 19860 consensus_queue.cc:1035] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.794246 19812 raft_consensus.cc:1270] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.796696 19877 consensus_queue.cc:1035] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.805383 19665 raft_consensus.cc:1270] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.808261 19877 consensus_queue.cc:1035] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.868917 19812 raft_consensus.cc:1270] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.869982 19860 consensus_queue.cc:1035] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.875633 19665 raft_consensus.cc:1270] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.877034 19861 consensus_queue.cc:1035] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.878716 19739 raft_consensus.cc:1270] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.879473 19740 raft_consensus.cc:1270] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.880540 19926 consensus_queue.cc:1035] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.881527 19861 consensus_queue.cc:1035] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.913926 19812 raft_consensus.cc:1270] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.915288 19877 consensus_queue.cc:1035] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.922012 19665 raft_consensus.cc:1270] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.924249 19877 consensus_queue.cc:1035] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:02.997103 19740 raft_consensus.cc:1270] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:02.998318 19926 consensus_queue.cc:1035] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:03.005934 19812 raft_consensus.cc:1270] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:03.007303 19860 consensus_queue.cc:1035] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:03.061362 19665 raft_consensus.cc:1270] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:03.062474 19877 consensus_queue.cc:1035] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:03.069447 19812 raft_consensus.cc:1270] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:03.071133 19862 consensus_queue.cc:1035] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:03.074560 19740 raft_consensus.cc:1270] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:03.076570 19860 consensus_queue.cc:1035] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:03.085610 19812 raft_consensus.cc:1270] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:03.087929 19896 consensus_queue.cc:1035] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:03.130934 19665 raft_consensus.cc:1270] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:03.132011 19862 consensus_queue.cc:1035] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:03.138988 19812 raft_consensus.cc:1270] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:03.140694 19933 consensus_queue.cc:1035] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:05.241389 15454 test_util.cc:274] Using random seed: -1725536603
I20250112 20:59:05.269542 19562 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58578:
name: "soft_deleted_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250112 20:59:05.272054 19562 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250112 20:59:05.318699 19648 tablet_service.cc:1467] Processing CreateTablet for tablet ffae6c92a63042b7b52e2c8c70d91648 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:05.319151 19726 tablet_service.cc:1467] Processing CreateTablet for tablet ffae6c92a63042b7b52e2c8c70d91648 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:05.320500 19648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffae6c92a63042b7b52e2c8c70d91648. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.320832 19726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffae6c92a63042b7b52e2c8c70d91648. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.321314 19653 tablet_service.cc:1467] Processing CreateTablet for tablet 395b5e250e9e40cdb1ba27ee7dca3cc7 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:05.322729 19653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395b5e250e9e40cdb1ba27ee7dca3cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.323648 19655 tablet_service.cc:1467] Processing CreateTablet for tablet 3226df3d47f748b28b4dcce8ea2f36e9 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:05.324899 19655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3226df3d47f748b28b4dcce8ea2f36e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.331405 19802 tablet_service.cc:1467] Processing CreateTablet for tablet ffae6c92a63042b7b52e2c8c70d91648 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:05.332119 19722 tablet_service.cc:1467] Processing CreateTablet for tablet 395b5e250e9e40cdb1ba27ee7dca3cc7 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:05.333896 19722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395b5e250e9e40cdb1ba27ee7dca3cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.335844 19798 tablet_service.cc:1467] Processing CreateTablet for tablet 395b5e250e9e40cdb1ba27ee7dca3cc7 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:05.337658 19798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 395b5e250e9e40cdb1ba27ee7dca3cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.340492 19802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffae6c92a63042b7b52e2c8c70d91648. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.365880 19730 tablet_service.cc:1467] Processing CreateTablet for tablet 3226df3d47f748b28b4dcce8ea2f36e9 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:05.367244 19730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3226df3d47f748b28b4dcce8ea2f36e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.369871 19804 tablet_service.cc:1467] Processing CreateTablet for tablet 3226df3d47f748b28b4dcce8ea2f36e9 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:05.371101 19804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3226df3d47f748b28b4dcce8ea2f36e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.378219 19647 tablet_service.cc:1467] Processing CreateTablet for tablet 55452388fe9a4cb38297efb9e7e0e54e (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:05.379544 19647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55452388fe9a4cb38297efb9e7e0e54e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.380849 19796 tablet_service.cc:1467] Processing CreateTablet for tablet 55452388fe9a4cb38297efb9e7e0e54e (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:05.382201 19796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55452388fe9a4cb38297efb9e7e0e54e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.382383 19729 tablet_service.cc:1467] Processing CreateTablet for tablet 55452388fe9a4cb38297efb9e7e0e54e (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:05.383725 19729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55452388fe9a4cb38297efb9e7e0e54e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.397867 19649 tablet_service.cc:1467] Processing CreateTablet for tablet ffbb7ca86d9745bf8365c49b9e629093 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:05.399191 19649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffbb7ca86d9745bf8365c49b9e629093. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.403780 19797 tablet_service.cc:1467] Processing CreateTablet for tablet ffbb7ca86d9745bf8365c49b9e629093 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:05.404165 19799 tablet_service.cc:1467] Processing CreateTablet for tablet 861abe0f055a430a950e7c7573db1947 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:05.404974 19652 tablet_service.cc:1467] Processing CreateTablet for tablet 861abe0f055a430a950e7c7573db1947 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:05.406272 19652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861abe0f055a430a950e7c7573db1947. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.409824 19654 tablet_service.cc:1467] Processing CreateTablet for tablet da5fcd0116034ac5939b328651511d10 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:05.411018 19654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5fcd0116034ac5939b328651511d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.412127 19797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffbb7ca86d9745bf8365c49b9e629093. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.413596 19799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861abe0f055a430a950e7c7573db1947. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.415073 19725 tablet_service.cc:1467] Processing CreateTablet for tablet ffbb7ca86d9745bf8365c49b9e629093 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:05.417732 19727 tablet_service.cc:1467] Processing CreateTablet for tablet da5fcd0116034ac5939b328651511d10 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:05.419131 19724 tablet_service.cc:1467] Processing CreateTablet for tablet 861abe0f055a430a950e7c7573db1947 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:05.425024 19727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5fcd0116034ac5939b328651511d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.425189 19803 tablet_service.cc:1467] Processing CreateTablet for tablet da5fcd0116034ac5939b328651511d10 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:05.426915 19803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5fcd0116034ac5939b328651511d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.427875 19724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 861abe0f055a430a950e7c7573db1947. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.429659 19725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffbb7ca86d9745bf8365c49b9e629093. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.449343 19723 tablet_service.cc:1467] Processing CreateTablet for tablet bbb4975e81294cf8a61f7c22304517f9 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:05.451040 19723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb4975e81294cf8a61f7c22304517f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.456483 19800 tablet_service.cc:1467] Processing CreateTablet for tablet bbb4975e81294cf8a61f7c22304517f9 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:05.458236 19800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb4975e81294cf8a61f7c22304517f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.462231 19801 tablet_service.cc:1467] Processing CreateTablet for tablet 179f24ca8e194189a8940dafe1c65418 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:05.463517 19801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179f24ca8e194189a8940dafe1c65418. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.462149 19958 tablet_bootstrap.cc:492] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:05.487030 19653 tablet_service.cc:1467] Processing CreateTablet for tablet 179f24ca8e194189a8940dafe1c65418 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:05.488442 19653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179f24ca8e194189a8940dafe1c65418. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.493956 19647 tablet_service.cc:1467] Processing CreateTablet for tablet bbb4975e81294cf8a61f7c22304517f9 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:05.498296 19958 tablet_bootstrap.cc:654] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.449806 19957 tablet_bootstrap.cc:492] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:05.509614 19722 tablet_service.cc:1467] Processing CreateTablet for tablet 179f24ca8e194189a8940dafe1c65418 (DEFAULT_TABLE table=soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:05.522562 19722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 179f24ca8e194189a8940dafe1c65418. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.525827 19959 tablet_bootstrap.cc:492] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.527395 19647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb4975e81294cf8a61f7c22304517f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:05.530781 19958 tablet_bootstrap.cc:492] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:05.541205 19958 ts_tablet_manager.cc:1397] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.079s user 0.014s sys 0.000s
I20250112 20:59:05.544164 19958 raft_consensus.cc:357] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.545415 19958 raft_consensus.cc:383] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.545763 19958 raft_consensus.cc:738] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.546540 19958 consensus_queue.cc:260] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.531976 19957 tablet_bootstrap.cc:654] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.549400 19958 ts_tablet_manager.cc:1428] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.008s user 0.000s sys 0.008s
I20250112 20:59:05.550315 19958 tablet_bootstrap.cc:492] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:05.556813 19958 tablet_bootstrap.cc:654] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.558390 19957 tablet_bootstrap.cc:492] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:05.558986 19957 ts_tablet_manager.cc:1397] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.110s user 0.011s sys 0.000s
I20250112 20:59:05.562501 19957 raft_consensus.cc:357] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.563184 19959 tablet_bootstrap.cc:654] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.563262 19957 raft_consensus.cc:383] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.563880 19957 raft_consensus.cc:738] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.564567 19957 consensus_queue.cc:260] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.567144 19957 ts_tablet_manager.cc:1428] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.008s user 0.000s sys 0.005s
I20250112 20:59:05.568734 19957 tablet_bootstrap.cc:492] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:05.570071 19958 tablet_bootstrap.cc:492] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:05.570899 19958 ts_tablet_manager.cc:1397] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.009s
I20250112 20:59:05.571012 19959 tablet_bootstrap.cc:492] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.571702 19959 ts_tablet_manager.cc:1397] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.046s user 0.014s sys 0.000s
I20250112 20:59:05.573976 19958 raft_consensus.cc:357] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.574780 19958 raft_consensus.cc:383] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.574561 19959 raft_consensus.cc:357] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.575129 19958 raft_consensus.cc:738] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.575516 19959 raft_consensus.cc:383] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.576032 19959 raft_consensus.cc:738] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.576293 19958 consensus_queue.cc:260] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.576921 19959 consensus_queue.cc:260] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.582137 19959 ts_tablet_manager.cc:1428] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20250112 20:59:05.582258 19957 tablet_bootstrap.cc:654] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.583226 19959 tablet_bootstrap.cc:492] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.582130 19958 ts_tablet_manager.cc:1428] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250112 20:59:05.588320 19958 tablet_bootstrap.cc:492] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:05.591835 19959 tablet_bootstrap.cc:654] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.594558 19958 tablet_bootstrap.cc:654] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.597321 19957 tablet_bootstrap.cc:492] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:05.597932 19957 ts_tablet_manager.cc:1397] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.013s
I20250112 20:59:05.601476 19957 raft_consensus.cc:357] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.602631 19957 raft_consensus.cc:383] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.603071 19957 raft_consensus.cc:738] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.604254 19957 consensus_queue.cc:260] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.610379 19957 ts_tablet_manager.cc:1428] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20250112 20:59:05.611595 19957 tablet_bootstrap.cc:492] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:05.622848 19957 tablet_bootstrap.cc:654] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
W20250112 20:59:05.628755 19611 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:59:05.633169 19959 tablet_bootstrap.cc:492] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.633322 19958 tablet_bootstrap.cc:492] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:05.633709 19959 ts_tablet_manager.cc:1397] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.051s user 0.019s sys 0.027s
I20250112 20:59:05.633826 19958 ts_tablet_manager.cc:1397] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.046s user 0.028s sys 0.017s
I20250112 20:59:05.634328 19957 tablet_bootstrap.cc:492] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:05.634824 19957 ts_tablet_manager.cc:1397] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.023s user 0.013s sys 0.003s
I20250112 20:59:05.636094 19958 raft_consensus.cc:357] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.636736 19958 raft_consensus.cc:383] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.637094 19958 raft_consensus.cc:738] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.636646 19959 raft_consensus.cc:357] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.637513 19959 raft_consensus.cc:383] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.637935 19959 raft_consensus.cc:738] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.637530 19957 raft_consensus.cc:357] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.637710 19958 consensus_queue.cc:260] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.638458 19957 raft_consensus.cc:383] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.638834 19957 raft_consensus.cc:738] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.638677 19959 consensus_queue.cc:260] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.639585 19957 consensus_queue.cc:260] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.641119 19959 ts_tablet_manager.cc:1428] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20250112 20:59:05.641988 19959 tablet_bootstrap.cc:492] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.642503 19958 ts_tablet_manager.cc:1428] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20250112 20:59:05.644984 19957 ts_tablet_manager.cc:1428] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20250112 20:59:05.646493 19958 tablet_bootstrap.cc:492] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:05.648454 19959 tablet_bootstrap.cc:654] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.646003 19957 tablet_bootstrap.cc:492] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:05.655397 19958 tablet_bootstrap.cc:654] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.658605 19957 tablet_bootstrap.cc:654] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.660032 19959 tablet_bootstrap.cc:492] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.660586 19959 ts_tablet_manager.cc:1397] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.012s
I20250112 20:59:05.664018 19959 raft_consensus.cc:357] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.665184 19959 raft_consensus.cc:383] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.665639 19959 raft_consensus.cc:738] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.666651 19959 consensus_queue.cc:260] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.669481 19959 ts_tablet_manager.cc:1428] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20250112 20:59:05.669492 19958 tablet_bootstrap.cc:492] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:05.670015 19957 tablet_bootstrap.cc:492] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:05.670264 19958 ts_tablet_manager.cc:1397] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.027s user 0.009s sys 0.012s
I20250112 20:59:05.670521 19957 ts_tablet_manager.cc:1397] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.025s user 0.015s sys 0.004s
I20250112 20:59:05.670604 19959 tablet_bootstrap.cc:492] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.673031 19958 raft_consensus.cc:357] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.673786 19958 raft_consensus.cc:383] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.673354 19957 raft_consensus.cc:357] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.674129 19958 raft_consensus.cc:738] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.674276 19957 raft_consensus.cc:383] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.674721 19957 raft_consensus.cc:738] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.674975 19958 consensus_queue.cc:260] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.675570 19957 consensus_queue.cc:260] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.677837 19959 tablet_bootstrap.cc:654] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.677951 19958 ts_tablet_manager.cc:1428] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250112 20:59:05.678969 19958 tablet_bootstrap.cc:492] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:05.681710 19957 ts_tablet_manager.cc:1428] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250112 20:59:05.682986 19957 tablet_bootstrap.cc:492] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:05.684927 19958 tablet_bootstrap.cc:654] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.689875 19959 tablet_bootstrap.cc:492] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.690368 19959 ts_tablet_manager.cc:1397] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20250112 20:59:05.690373 19957 tablet_bootstrap.cc:654] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.693468 19959 raft_consensus.cc:357] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.694489 19959 raft_consensus.cc:383] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.694865 19959 raft_consensus.cc:738] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.695993 19877 raft_consensus.cc:491] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.696547 19877 raft_consensus.cc:513] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.699940 19877 leader_election.cc:290] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.699980 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffae6c92a63042b7b52e2c8c70d91648" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:05.700809 19959 consensus_queue.cc:260] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.702070 19957 tablet_bootstrap.cc:492] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:05.702553 19957 ts_tablet_manager.cc:1397] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.004s
I20250112 20:59:05.703467 19959 ts_tablet_manager.cc:1428] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20250112 20:59:05.703147 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffae6c92a63042b7b52e2c8c70d91648" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:05.704120 19812 raft_consensus.cc:2463] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 0.
I20250112 20:59:05.701112 19665 raft_consensus.cc:2463] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 0.
I20250112 20:59:05.705111 19957 raft_consensus.cc:357] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.705942 19957 raft_consensus.cc:383] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.705787 19703 leader_election.cc:304] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:05.706226 19957 raft_consensus.cc:738] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.706933 19877 raft_consensus.cc:2798] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:05.707809 19877 raft_consensus.cc:491] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:05.708226 19877 raft_consensus.cc:3054] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.711011 19959 tablet_bootstrap.cc:492] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.715873 19877 raft_consensus.cc:513] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.717801 19959 tablet_bootstrap.cc:654] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.717124 19957 consensus_queue.cc:260] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.721912 19957 ts_tablet_manager.cc:1428] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20250112 20:59:05.723284 19877 leader_election.cc:290] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.724133 19957 tablet_bootstrap.cc:492] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:05.724336 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffae6c92a63042b7b52e2c8c70d91648" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:05.724963 19812 raft_consensus.cc:3054] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.730450 19957 tablet_bootstrap.cc:654] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.732239 19812 raft_consensus.cc:2463] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 1.
I20250112 20:59:05.733253 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffae6c92a63042b7b52e2c8c70d91648" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:05.733490 19703 leader_election.cc:304] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:05.734269 19941 raft_consensus.cc:491] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.734593 19877 raft_consensus.cc:2798] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:05.734894 19941 raft_consensus.cc:513] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.736146 19877 raft_consensus.cc:695] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:05.737157 19877 consensus_queue.cc:237] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.738503 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55452388fe9a4cb38297efb9e7e0e54e" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:05.739146 19740 raft_consensus.cc:2463] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:05.740485 19665 raft_consensus.cc:3054] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.740566 19628 leader_election.cc:304] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:05.742125 19941 leader_election.cc:290] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.742202 19959 tablet_bootstrap.cc:492] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.742717 19941 raft_consensus.cc:2798] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:05.742847 19959 ts_tablet_manager.cc:1397] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.032s user 0.017s sys 0.000s
I20250112 20:59:05.743106 19941 raft_consensus.cc:491] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:05.743485 19941 raft_consensus.cc:3054] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.749213 19959 raft_consensus.cc:357] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.750268 19959 raft_consensus.cc:383] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.750665 19959 raft_consensus.cc:738] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.751777 19665 raft_consensus.cc:2463] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 1.
I20250112 20:59:05.751703 19959 consensus_queue.cc:260] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.754997 19957 tablet_bootstrap.cc:492] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:05.755059 19959 ts_tablet_manager.cc:1428] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20250112 20:59:05.760954 19957 ts_tablet_manager.cc:1397] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.004s
I20250112 20:59:05.760098 19562 catalog_manager.cc:5526] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:05.761497 19959 tablet_bootstrap.cc:492] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.764415 19957 raft_consensus.cc:357] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.765918 19957 raft_consensus.cc:383] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.766237 19957 raft_consensus.cc:738] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.767114 19957 consensus_queue.cc:260] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.768661 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55452388fe9a4cb38297efb9e7e0e54e" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:05.769205 19959 tablet_bootstrap.cc:654] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
W20250112 20:59:05.770126 19628 leader_election.cc:343] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.770614 19957 ts_tablet_manager.cc:1428] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250112 20:59:05.759145 19958 tablet_bootstrap.cc:492] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:05.771471 19958 ts_tablet_manager.cc:1397] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.093s user 0.029s sys 0.040s
I20250112 20:59:05.771677 19957 tablet_bootstrap.cc:492] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:05.774298 19958 raft_consensus.cc:357] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.775045 19958 raft_consensus.cc:383] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.775362 19958 raft_consensus.cc:738] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.776180 19958 consensus_queue.cc:260] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.778838 19958 ts_tablet_manager.cc:1428] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20250112 20:59:05.779807 19958 tablet_bootstrap.cc:492] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:05.784282 19959 tablet_bootstrap.cc:492] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.784546 19957 tablet_bootstrap.cc:654] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.784924 19959 ts_tablet_manager.cc:1397] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.024s user 0.015s sys 0.003s
I20250112 20:59:05.786144 19958 tablet_bootstrap.cc:654] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.786841 19978 raft_consensus.cc:491] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.787348 19978 raft_consensus.cc:513] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.789614 19978 leader_election.cc:290] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.789683 19959 raft_consensus.cc:357] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.790880 19959 raft_consensus.cc:383] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.791282 19959 raft_consensus.cc:738] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.791882 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3226df3d47f748b28b4dcce8ea2f36e9" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:05.792758 19812 raft_consensus.cc:2463] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:05.792164 19959 consensus_queue.cc:260] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.794298 19861 raft_consensus.cc:491] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.794857 19861 raft_consensus.cc:513] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.797537 19861 leader_election.cc:290] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:05.800159 19959 ts_tablet_manager.cc:1428] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20250112 20:59:05.800567 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:05.801244 19959 tablet_bootstrap.cc:492] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.801573 19958 tablet_bootstrap.cc:492] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:05.802050 19958 ts_tablet_manager.cc:1397] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.008s
W20250112 20:59:05.802101 19780 leader_election.cc:343] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.803810 19628 leader_election.cc:304] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:05.805020 19958 raft_consensus.cc:357] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.805946 19958 raft_consensus.cc:383] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.806265 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3226df3d47f748b28b4dcce8ea2f36e9" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:05.806950 19739 raft_consensus.cc:2463] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:05.807030 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:05.806293 19958 raft_consensus.cc:738] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.808132 19959 tablet_bootstrap.cc:654] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
W20250112 20:59:05.808552 19777 leader_election.cc:343] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.809062 19777 leader_election.cc:304] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9; no voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740
I20250112 20:59:05.808738 19958 consensus_queue.cc:260] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.810259 19861 raft_consensus.cc:2743] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:05.817987 19978 raft_consensus.cc:491] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.818562 19978 raft_consensus.cc:513] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.819775 19958 ts_tablet_manager.cc:1428] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.017s user 0.007s sys 0.006s
I20250112 20:59:05.820773 19958 tablet_bootstrap.cc:492] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:05.826517 19959 tablet_bootstrap.cc:492] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.827136 19958 tablet_bootstrap.cc:654] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.827241 19959 ts_tablet_manager.cc:1397] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.006s
I20250112 20:59:05.829308 19978 leader_election.cc:290] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.830384 19978 raft_consensus.cc:2798] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:05.830998 19985 raft_consensus.cc:491] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.831514 19978 raft_consensus.cc:491] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:05.831125 19959 raft_consensus.cc:357] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.832330 19959 raft_consensus.cc:383] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.831938 19978 raft_consensus.cc:3054] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.831660 19985 raft_consensus.cc:513] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.832744 19959 raft_consensus.cc:738] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.833879 19959 consensus_queue.cc:260] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.834920 19983 raft_consensus.cc:491] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.835404 19983 raft_consensus.cc:513] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.836859 19959 ts_tablet_manager.cc:1428] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20250112 20:59:05.837308 19983 leader_election.cc:290] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:05.837764 19959 tablet_bootstrap.cc:492] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.838788 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:05.839450 19740 raft_consensus.cc:2463] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
I20250112 20:59:05.839665 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:05.840240 19665 raft_consensus.cc:2463] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
I20250112 20:59:05.840659 19777 leader_election.cc:304] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:05.841948 19983 raft_consensus.cc:2798] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:05.842358 19983 raft_consensus.cc:491] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:05.842753 19983 raft_consensus.cc:3054] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.844990 19959 tablet_bootstrap.cc:654] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
W20250112 20:59:05.847594 19628 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.23.131:41093, user_credentials={real_user=slave}} blocked reactor thread for 43881.9us
I20250112 20:59:05.849179 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:05.849866 19740 raft_consensus.cc:2463] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:05.851627 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:05.851840 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179f24ca8e194189a8940dafe1c65418" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:05.850463 19983 raft_consensus.cc:513] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.853785 19812 raft_consensus.cc:2388] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f0b8ad29373745ddac1bcb23ca06cf25 in current term 1: Already voted for candidate b24e8e78a6834a6fae06c193b5ac43a9 in this term.
I20250112 20:59:05.853871 19628 leader_election.cc:304] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
W20250112 20:59:05.854729 19628 leader_election.cc:343] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.856280 19959 tablet_bootstrap.cc:492] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.856369 19861 raft_consensus.cc:491] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.857518 19959 ts_tablet_manager.cc:1397] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.012s
I20250112 20:59:05.834424 19985 leader_election.cc:290] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.857560 19861 raft_consensus.cc:513] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.858243 19985 raft_consensus.cc:2798] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:05.858546 19985 raft_consensus.cc:491] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:05.858840 19985 raft_consensus.cc:3054] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.859535 19861 leader_election.cc:290] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:05.859999 19959 raft_consensus.cc:357] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.855923 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179f24ca8e194189a8940dafe1c65418" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:05.860672 19959 raft_consensus.cc:383] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.860981 19812 raft_consensus.cc:2463] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:05.861106 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:05.861774 19664 raft_consensus.cc:2463] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
I20250112 20:59:05.860973 19959 raft_consensus.cc:738] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.856926 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:05.862116 19628 leader_election.cc:304] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters: f9d40940ddca458db0f091c07f19b740
I20250112 20:59:05.862529 19959 consensus_queue.cc:260] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.863662 19780 leader_election.cc:304] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:05.864506 19861 raft_consensus.cc:2798] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:05.864630 19959 ts_tablet_manager.cc:1428] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250112 20:59:05.864890 19861 raft_consensus.cc:491] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:05.865267 19861 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.865543 19959 tablet_bootstrap.cc:492] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740: Bootstrap starting.
I20250112 20:59:05.867063 19978 raft_consensus.cc:513] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.863248 19982 raft_consensus.cc:2798] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:05.868911 19982 raft_consensus.cc:491] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:05.869055 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:05.759657 19941 raft_consensus.cc:513] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.869669 19740 raft_consensus.cc:2463] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
I20250112 20:59:05.871675 19959 tablet_bootstrap.cc:654] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.869272 19982 raft_consensus.cc:3054] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.874624 19861 raft_consensus.cc:513] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.876022 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55452388fe9a4cb38297efb9e7e0e54e" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:05.876688 19740 raft_consensus.cc:3054] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.877282 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:05.877888 19664 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.880654 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55452388fe9a4cb38297efb9e7e0e54e" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:05.885782 19664 raft_consensus.cc:2463] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 1.
I20250112 20:59:05.886988 19780 leader_election.cc:304] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:05.873695 19941 leader_election.cc:290] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.897424 19861 leader_election.cc:290] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:05.887883 19982 raft_consensus.cc:513] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
W20250112 20:59:05.899443 19628 leader_election.cc:343] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.908977 19877 raft_consensus.cc:491] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.899574 19738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:05.909354 19877 raft_consensus.cc:513] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.893548 19959 tablet_bootstrap.cc:492] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740: No bootstrap required, opened a new log
I20250112 20:59:05.898478 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:05.910178 19959 ts_tablet_manager.cc:1397] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.016s
I20250112 20:59:05.909495 19738 raft_consensus.cc:3054] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.910526 19739 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.911156 19877 leader_election.cc:290] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.912568 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5fcd0116034ac5939b328651511d10" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:05.912799 19959 raft_consensus.cc:357] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.913436 19959 raft_consensus.cc:383] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.913686 19959 raft_consensus.cc:738] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Initialized, Role: FOLLOWER
W20250112 20:59:05.914245 19703 leader_election.cc:343] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.914278 19959 consensus_queue.cc:260] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.915357 19877 raft_consensus.cc:491] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.915822 19877 raft_consensus.cc:513] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.904896 19978 leader_election.cc:290] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.916832 19959 ts_tablet_manager.cc:1428] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250112 20:59:05.902765 19958 tablet_bootstrap.cc:492] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:05.917547 19958 ts_tablet_manager.cc:1397] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.097s user 0.012s sys 0.004s
I20250112 20:59:05.917984 19739 raft_consensus.cc:2463] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 1.
I20250112 20:59:05.909485 19957 tablet_bootstrap.cc:492] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:05.897975 19983 leader_election.cc:290] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:05.920348 19958 raft_consensus.cc:357] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.920899 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5fcd0116034ac5939b328651511d10" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:05.921360 19958 raft_consensus.cc:383] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.921689 19958 raft_consensus.cc:738] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
W20250112 20:59:05.922421 19706 leader_election.cc:343] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.922947 19706 leader_election.cc:304] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:05.904474 19989 raft_consensus.cc:2798] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:05.923650 19989 raft_consensus.cc:695] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:05.919221 19957 ts_tablet_manager.cc:1397] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.148s user 0.029s sys 0.035s
I20250112 20:59:05.924408 19989 consensus_queue.cc:237] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.926113 19738 raft_consensus.cc:2463] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 1.
I20250112 20:59:05.922500 19958 consensus_queue.cc:260] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.924875 19976 raft_consensus.cc:2743] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:05.927754 19777 leader_election.cc:304] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:05.926983 19957 raft_consensus.cc:357] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.928647 19957 raft_consensus.cc:383] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:05.928742 19983 raft_consensus.cc:2798] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:05.928449 19985 raft_consensus.cc:513] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.929066 19957 raft_consensus.cc:738] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:05.929316 19983 raft_consensus.cc:695] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:05.929880 19957 consensus_queue.cc:260] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.930454 19665 raft_consensus.cc:2388] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b24e8e78a6834a6fae06c193b5ac43a9 in current term 1: Already voted for candidate f0b8ad29373745ddac1bcb23ca06cf25 in this term.
I20250112 20:59:05.930133 19983 consensus_queue.cc:237] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.932365 19957 ts_tablet_manager.cc:1428] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20250112 20:59:05.932365 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:05.933580 19957 tablet_bootstrap.cc:492] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:05.934109 19877 leader_election.cc:290] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.934206 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3226df3d47f748b28b4dcce8ea2f36e9" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:05.934832 19812 raft_consensus.cc:3054] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.934899 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3226df3d47f748b28b4dcce8ea2f36e9" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:05.935509 19739 raft_consensus.cc:3054] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:59:05.935698 19706 leader_election.cc:343] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.937454 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
W20250112 20:59:05.938730 19703 leader_election.cc:343] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:05.939168 19703 leader_election.cc:304] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:05.907205 19740 raft_consensus.cc:2463] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:05.940379 19957 tablet_bootstrap.cc:654] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.940615 19738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:05.941329 19738 raft_consensus.cc:2388] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f0b8ad29373745ddac1bcb23ca06cf25 in current term 1: Already voted for candidate b24e8e78a6834a6fae06c193b5ac43a9 in this term.
I20250112 20:59:05.941210 19628 leader_election.cc:304] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters: b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:05.943087 19877 raft_consensus.cc:2743] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:05.944787 19958 ts_tablet_manager.cc:1428] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.027s user 0.003s sys 0.005s
I20250112 20:59:05.945853 19958 tablet_bootstrap.cc:492] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:05.946853 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:05.948283 19628 leader_election.cc:304] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25; no voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740
I20250112 20:59:05.949397 19990 raft_consensus.cc:2798] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:05.949899 19990 raft_consensus.cc:695] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:05.950047 19941 raft_consensus.cc:2743] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250112 20:59:05.952098 19958 tablet_bootstrap.cc:654] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:05.952817 19982 leader_election.cc:290] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.942544 19985 leader_election.cc:290] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.957237 19739 raft_consensus.cc:2463] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:05.950742 19990 consensus_queue.cc:237] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.960990 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179f24ca8e194189a8940dafe1c65418" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:05.961791 19740 raft_consensus.cc:3054] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.961964 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179f24ca8e194189a8940dafe1c65418" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:05.962822 19813 raft_consensus.cc:3054] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.970609 19812 raft_consensus.cc:2463] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:05.976145 19628 leader_election.cc:304] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:05.977989 19978 raft_consensus.cc:2798] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:05.978435 19740 raft_consensus.cc:2463] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:05.979005 19978 raft_consensus.cc:695] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:05.984088 19985 raft_consensus.cc:491] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:05.985759 19985 raft_consensus.cc:513] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.988325 19985 leader_election.cc:290] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:05.990015 19978 consensus_queue.cc:237] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:05.992359 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:05.996941 19562 catalog_manager.cc:5526] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:05.998062 19739 raft_consensus.cc:2463] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:05.993698 19628 leader_election.cc:304] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:05.999291 19562 catalog_manager.cc:5526] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:06.000244 19985 raft_consensus.cc:2798] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:06.000859 19985 raft_consensus.cc:695] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:06.001281 19628 leader_election.cc:304] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.002105 19982 raft_consensus.cc:2798] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:06.001843 19985 consensus_queue.cc:237] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.002557 19982 raft_consensus.cc:491] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:06.003031 19982 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:05.974354 19813 raft_consensus.cc:2463] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:06.008916 19957 tablet_bootstrap.cc:492] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:06.010332 19957 ts_tablet_manager.cc:1397] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.077s user 0.027s sys 0.045s
I20250112 20:59:06.011897 19983 raft_consensus.cc:491] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:06.012338 19983 raft_consensus.cc:513] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.013231 19957 raft_consensus.cc:357] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.013906 19957 raft_consensus.cc:383] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:06.014250 19983 leader_election.cc:290] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:06.014950 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55452388fe9a4cb38297efb9e7e0e54e" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:06.015380 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55452388fe9a4cb38297efb9e7e0e54e" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:06.016501 19740 raft_consensus.cc:2388] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b24e8e78a6834a6fae06c193b5ac43a9 in current term 1: Already voted for candidate f0b8ad29373745ddac1bcb23ca06cf25 in this term.
I20250112 20:59:06.017316 19982 raft_consensus.cc:513] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.017697 19777 leader_election.cc:304] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9; no voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740
I20250112 20:59:06.019479 19983 raft_consensus.cc:3054] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.020093 19958 tablet_bootstrap.cc:492] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:06.020537 19958 ts_tablet_manager.cc:1397] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.075s user 0.014s sys 0.041s
I20250112 20:59:06.014158 19957 raft_consensus.cc:738] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:06.021647 19957 consensus_queue.cc:260] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.025736 19958 raft_consensus.cc:357] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.026440 19958 raft_consensus.cc:383] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:06.026738 19958 raft_consensus.cc:738] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:06.027319 19958 consensus_queue.cc:260] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.029121 19957 ts_tablet_manager.cc:1428] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.018s user 0.004s sys 0.002s
I20250112 20:59:06.029809 19958 ts_tablet_manager.cc:1428] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.009s user 0.003s sys 0.001s
I20250112 20:59:06.018302 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:06.030136 19957 tablet_bootstrap.cc:492] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25: Bootstrap starting.
I20250112 20:59:06.030586 19958 tablet_bootstrap.cc:492] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9: Bootstrap starting.
I20250112 20:59:06.030578 19812 raft_consensus.cc:2463] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:06.032569 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:06.033190 19740 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.036274 19958 tablet_bootstrap.cc:654] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:06.036207 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:06.037086 19813 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.040114 19740 raft_consensus.cc:2463] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:06.044802 19628 leader_election.cc:304] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.045584 19941 raft_consensus.cc:2798] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:06.046047 19941 raft_consensus.cc:695] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:06.042680 19983 raft_consensus.cc:2743] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:06.043049 19982 leader_election.cc:290] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.048247 19941 consensus_queue.cc:237] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.051859 19957 tablet_bootstrap.cc:654] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:06.055819 19813 raft_consensus.cc:2463] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:06.057612 19562 catalog_manager.cc:5526] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.058858 19562 catalog_manager.cc:5526] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.059667 19958 tablet_bootstrap.cc:492] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9: No bootstrap required, opened a new log
I20250112 20:59:06.060091 19958 ts_tablet_manager.cc:1397] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.015s
I20250112 20:59:06.062047 19957 tablet_bootstrap.cc:492] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25: No bootstrap required, opened a new log
I20250112 20:59:06.062485 19957 ts_tablet_manager.cc:1397] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent bootstrapping tablet: real 0.033s user 0.005s sys 0.005s
I20250112 20:59:06.062709 19958 raft_consensus.cc:357] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.063381 19958 raft_consensus.cc:383] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:06.063625 19958 raft_consensus.cc:738] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Initialized, Role: FOLLOWER
I20250112 20:59:06.064159 19958 consensus_queue.cc:260] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.065125 19957 raft_consensus.cc:357] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.065819 19957 raft_consensus.cc:383] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:06.066057 19957 raft_consensus.cc:738] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Initialized, Role: FOLLOWER
I20250112 20:59:06.066183 19958 ts_tablet_manager.cc:1428] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20250112 20:59:06.066738 19957 consensus_queue.cc:260] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.068436 19957 ts_tablet_manager.cc:1428] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20250112 20:59:06.079746 19562 catalog_manager.cc:5526] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.081053 19562 catalog_manager.cc:5526] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.134591 19983 raft_consensus.cc:491] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:06.135191 19983 raft_consensus.cc:513] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.137092 19999 raft_consensus.cc:491] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:06.137290 19983 leader_election.cc:290] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:06.137581 19999 raft_consensus.cc:513] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.138056 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5fcd0116034ac5939b328651511d10" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" is_pre_election: true
I20250112 20:59:06.138669 19664 raft_consensus.cc:2463] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
I20250112 20:59:06.138762 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5fcd0116034ac5939b328651511d10" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:06.139343 19740 raft_consensus.cc:2463] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 0.
I20250112 20:59:06.139747 19780 leader_election.cc:304] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:06.139379 19999 leader_election.cc:290] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.140462 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740" is_pre_election: true
I20250112 20:59:06.140733 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" is_pre_election: true
I20250112 20:59:06.140755 19983 raft_consensus.cc:2798] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:06.141089 19740 raft_consensus.cc:2463] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:06.141220 19813 raft_consensus.cc:2463] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 0.
I20250112 20:59:06.141237 19983 raft_consensus.cc:491] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:06.141748 19983 raft_consensus.cc:3054] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.142030 19628 leader_election.cc:304] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.142596 19999 raft_consensus.cc:2798] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:06.142963 19999 raft_consensus.cc:491] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:06.143263 19999 raft_consensus.cc:3054] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.146735 19983 raft_consensus.cc:513] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.147782 19999 raft_consensus.cc:513] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.148080 19983 leader_election.cc:290] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:06.148952 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5fcd0116034ac5939b328651511d10" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:06.149592 19664 raft_consensus.cc:3054] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.149888 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5fcd0116034ac5939b328651511d10" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:06.150465 19740 raft_consensus.cc:3054] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.151087 19999 leader_election.cc:290] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.153445 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:06.153501 19877 consensus_queue.cc:1035] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:59:06.154212 19739 raft_consensus.cc:3054] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.154300 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:06.154994 19812 raft_consensus.cc:3054] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:06.156100 19664 raft_consensus.cc:2463] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 1.
I20250112 20:59:06.157255 19780 leader_election.cc:304] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:06.158272 19989 raft_consensus.cc:2798] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:06.158731 19989 raft_consensus.cc:695] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:06.159515 19989 consensus_queue.cc:237] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.161500 19812 raft_consensus.cc:2463] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:06.162438 19628 leader_election.cc:304] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:06.163373 19999 raft_consensus.cc:2798] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:06.163805 19999 raft_consensus.cc:695] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:06.164525 19999 consensus_queue.cc:237] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.166303 19739 raft_consensus.cc:2463] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 1.
I20250112 20:59:06.164913 19740 raft_consensus.cc:2463] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 1.
I20250112 20:59:06.178510 19560 catalog_manager.cc:5526] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.180673 19877 consensus_queue.cc:1035] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.183606 19562 catalog_manager.cc:5526] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:06.242090 19739 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ffae6c92a63042b7b52e2c8c70d91648"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
mode: GRACEFUL
new_leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:46774
I20250112 20:59:06.242635 19739 raft_consensus.cc:604] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Received request to transfer leadership to TS f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:06.247298 19813 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
mode: GRACEFUL
new_leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:57364
I20250112 20:59:06.247874 19813 raft_consensus.cc:604] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Received request to transfer leadership to TS f0b8ad29373745ddac1bcb23ca06cf25
I20250112 20:59:06.259003 19664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
mode: GRACEFUL
new_leader_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:32898
I20250112 20:59:06.259541 19664 raft_consensus.cc:604] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Received request to transfer leadership to TS f9d40940ddca458db0f091c07f19b740
I20250112 20:59:06.264726 19813 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "861abe0f055a430a950e7c7573db1947"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
mode: GRACEFUL
new_leader_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:57364
I20250112 20:59:06.265292 19813 raft_consensus.cc:604] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Received request to transfer leadership to TS f9d40940ddca458db0f091c07f19b740
I20250112 20:59:06.269835 19813 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "da5fcd0116034ac5939b328651511d10"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
mode: GRACEFUL
new_leader_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:57364
I20250112 20:59:06.270368 19813 raft_consensus.cc:604] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Received request to transfer leadership to TS f9d40940ddca458db0f091c07f19b740
I20250112 20:59:06.275084 19664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
mode: GRACEFUL
new_leader_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:32898
I20250112 20:59:06.275575 19664 raft_consensus.cc:604] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Received request to transfer leadership to TS f9d40940ddca458db0f091c07f19b740
I20250112 20:59:06.280427 19664 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "179f24ca8e194189a8940dafe1c65418"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
mode: GRACEFUL
new_leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:32898
I20250112 20:59:06.281075 19664 raft_consensus.cc:604] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Received request to transfer leadership to TS b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:06.319315 19989 consensus_queue.cc:1035] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.328693 19991 consensus_queue.cc:1035] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:59:06.338646 19861 raft_consensus.cc:988] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9: : Instructing follower f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:06.339076 19861 raft_consensus.cc:1076] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Signalling peer f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:06.340549 19664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:32896
I20250112 20:59:06.341111 19664 raft_consensus.cc:491] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:06.341388 19664 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.346069 19664 raft_consensus.cc:513] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.347491 19664 leader_election.cc:290] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.348193 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:06.348372 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:06.348788 19739 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.348997 19813 raft_consensus.cc:3049] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:06.349301 19813 raft_consensus.cc:738] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:06.349913 19813 consensus_queue.cc:260] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.350755 19813 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.353262 19739 raft_consensus.cc:2463] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:06.354175 19628 leader_election.cc:304] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.354929 19999 raft_consensus.cc:2798] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:06.355367 19999 raft_consensus.cc:695] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:06.355472 19991 consensus_queue.cc:1035] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:59:06.356007 19813 raft_consensus.cc:2463] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:06.356052 19999 consensus_queue.cc:237] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.365999 19999 consensus_queue.cc:1035] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.366497 19991 consensus_queue.cc:1035] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.364221 19562 catalog_manager.cc:5526] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 1 to 2, leader changed from b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131) to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.378170 19991 raft_consensus.cc:988] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9: : Instructing follower f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:06.378636 19983 raft_consensus.cc:1076] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Signalling peer f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:06.387315 19941 consensus_queue.cc:1035] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.390884 19739 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "861abe0f055a430a950e7c7573db1947"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:46760
I20250112 20:59:06.391628 19739 raft_consensus.cc:491] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:06.392092 19739 raft_consensus.cc:3054] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.401031 19739 raft_consensus.cc:513] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.403342 19739 leader_election.cc:290] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.403683 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:06.403970 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861abe0f055a430a950e7c7573db1947" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:06.404623 19664 raft_consensus.cc:3054] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.404664 19813 raft_consensus.cc:3049] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:06.405236 19813 raft_consensus.cc:738] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:06.406003 19813 consensus_queue.cc:260] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.407102 19813 raft_consensus.cc:3054] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.411232 19664 raft_consensus.cc:2463] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:06.412174 19706 leader_election.cc:304] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.412812 19976 raft_consensus.cc:2798] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:06.412854 19813 raft_consensus.cc:2463] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:06.413322 19976 raft_consensus.cc:695] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:06.414019 19976 consensus_queue.cc:237] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.416368 19999 consensus_queue.cc:1035] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.421486 19563 catalog_manager.cc:5526] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 1 to 2, leader changed from b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131) to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.425967 19982 consensus_queue.cc:1035] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.428541 19978 consensus_queue.cc:1035] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.452136 19978 consensus_queue.cc:1035] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.463757 19978 raft_consensus.cc:988] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25: : Instructing follower b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:06.464174 19999 raft_consensus.cc:1076] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Signalling peer b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:06.465440 19813 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "179f24ca8e194189a8940dafe1c65418"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:57334
I20250112 20:59:06.465847 19813 raft_consensus.cc:491] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:06.466096 19813 raft_consensus.cc:3054] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.470093 19813 raft_consensus.cc:513] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.471297 19813 leader_election.cc:290] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:06.472090 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179f24ca8e194189a8940dafe1c65418" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:06.472313 19740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "179f24ca8e194189a8940dafe1c65418" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:06.472692 19664 raft_consensus.cc:3049] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:06.472932 19740 raft_consensus.cc:3054] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.472985 19664 raft_consensus.cc:738] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:06.473683 19664 consensus_queue.cc:260] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.474550 19664 raft_consensus.cc:3054] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.477200 19740 raft_consensus.cc:2463] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 2.
I20250112 20:59:06.478015 19777 leader_election.cc:304] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.478745 19991 raft_consensus.cc:2798] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:06.479189 19991 raft_consensus.cc:695] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:06.479398 19664 raft_consensus.cc:2463] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 2.
I20250112 20:59:06.480067 19991 consensus_queue.cc:237] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.486470 19563 catalog_manager.cc:5526] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 1 to 2, leader changed from f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129) to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 2 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:06.559120 19991 consensus_queue.cc:1035] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.564509 19861 raft_consensus.cc:988] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9: : Instructing follower f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:06.565016 19983 raft_consensus.cc:1076] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Signalling peer f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:06.566515 19739 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da5fcd0116034ac5939b328651511d10"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:46760
I20250112 20:59:06.567454 19739 raft_consensus.cc:491] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:06.567814 19739 raft_consensus.cc:3054] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.570036 19983 consensus_queue.cc:1035] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.574138 19739 raft_consensus.cc:513] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.576694 19665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5fcd0116034ac5939b328651511d10" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:06.576977 19739 leader_election.cc:290] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.577234 19665 raft_consensus.cc:3054] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.580410 19813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da5fcd0116034ac5939b328651511d10" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:06.581035 19813 raft_consensus.cc:3049] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:06.581276 19813 raft_consensus.cc:738] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:06.581961 19813 consensus_queue.cc:260] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.583032 19813 raft_consensus.cc:3054] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.583340 19665 raft_consensus.cc:2463] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:06.584288 19706 leader_election.cc:304] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.585042 19976 raft_consensus.cc:2798] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:06.585484 19976 raft_consensus.cc:695] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:06.586272 19976 consensus_queue.cc:237] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.589027 19813 raft_consensus.cc:2463] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:06.594619 19560 catalog_manager.cc:5526] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 1 to 2, leader changed from b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131) to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.596515 19999 consensus_queue.cc:1035] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.605895 19982 consensus_queue.cc:1035] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.613517 19999 raft_consensus.cc:988] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25: : Instructing follower f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:06.613940 19999 raft_consensus.cc:1076] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Signalling peer f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:06.615260 19739 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:46744
I20250112 20:59:06.615823 19739 raft_consensus.cc:491] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:06.616146 19739 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.617254 19877 raft_consensus.cc:988] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740: : Instructing follower f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:06.617708 19877 raft_consensus.cc:1076] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Signalling peer f0b8ad29373745ddac1bcb23ca06cf25 to start an election
I20250112 20:59:06.618880 19982 consensus_queue.cc:1035] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.619187 19665 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ffae6c92a63042b7b52e2c8c70d91648"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
from {username='slave'} at 127.0.0.1:32882
I20250112 20:59:06.619676 19665 raft_consensus.cc:491] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:06.619978 19665 raft_consensus.cc:3054] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.621398 19739 raft_consensus.cc:513] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.623152 19739 leader_election.cc:290] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.624078 19812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:06.624099 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:06.624732 19812 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.624868 19664 raft_consensus.cc:3049] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:06.625295 19664 raft_consensus.cc:738] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:06.626013 19664 consensus_queue.cc:260] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.627084 19664 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.627104 19665 raft_consensus.cc:513] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.629096 19665 leader_election.cc:290] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Requested vote from peers f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.630149 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffae6c92a63042b7b52e2c8c70d91648" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:06.636989 19814 raft_consensus.cc:3054] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.633664 19664 raft_consensus.cc:2463] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:06.638352 19706 leader_election.cc:304] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.634276 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffae6c92a63042b7b52e2c8c70d91648" candidate_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:06.639103 19976 raft_consensus.cc:2798] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:06.639235 19739 raft_consensus.cc:3049] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:06.639632 19976 raft_consensus.cc:695] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:06.635057 19812 raft_consensus.cc:2463] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:06.639683 19739 raft_consensus.cc:738] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:06.640504 19976 consensus_queue.cc:237] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.641541 19739 consensus_queue.cc:260] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.642611 19739 raft_consensus.cc:3054] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.645972 19999 consensus_queue.cc:1035] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:06.648485 19739 raft_consensus.cc:2463] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:06.649904 19628 leader_election.cc:304] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.650967 19999 raft_consensus.cc:2798] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:06.651407 19999 raft_consensus.cc:695] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Becoming Leader. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:06.652088 19999 consensus_queue.cc:237] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.654647 19563 catalog_manager.cc:5526] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 1 to 2, leader changed from f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129) to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.656558 19814 raft_consensus.cc:2463] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0b8ad29373745ddac1bcb23ca06cf25 in term 2.
I20250112 20:59:06.664072 19999 raft_consensus.cc:988] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25: : Instructing follower f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:06.664448 19999 raft_consensus.cc:1076] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Signalling peer f9d40940ddca458db0f091c07f19b740 to start an election
I20250112 20:59:06.664671 19562 catalog_manager.cc:5526] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 reported cstate change: term changed from 1 to 2, leader changed from f9d40940ddca458db0f091c07f19b740 (127.15.23.130) to f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.666592 19740 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
from {username='slave'} at 127.0.0.1:46744
I20250112 20:59:06.667148 19740 raft_consensus.cc:491] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:06.667435 19740 raft_consensus.cc:3054] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.672971 19740 raft_consensus.cc:513] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.674589 19740 leader_election.cc:290] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131:41093)
I20250112 20:59:06.675201 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:06.675661 19814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbb4975e81294cf8a61f7c22304517f9" candidate_uuid: "f9d40940ddca458db0f091c07f19b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
I20250112 20:59:06.675837 19664 raft_consensus.cc:3049] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:06.676198 19664 raft_consensus.cc:738] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:06.676982 19664 consensus_queue.cc:260] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.677914 19664 raft_consensus.cc:3054] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.683027 19664 raft_consensus.cc:2463] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:06.683661 19814 raft_consensus.cc:3054] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:06.683854 19706 leader_election.cc:304] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f0b8ad29373745ddac1bcb23ca06cf25, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:06.684715 19976 raft_consensus.cc:2798] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:06.685187 19976 raft_consensus.cc:695] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Leader. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:06.685979 19976 consensus_queue.cc:237] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:06.689600 19814 raft_consensus.cc:2463] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d40940ddca458db0f091c07f19b740 in term 2.
I20250112 20:59:06.692552 19562 catalog_manager.cc:5526] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 reported cstate change: term changed from 1 to 2, leader changed from f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129) to f9d40940ddca458db0f091c07f19b740 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "f9d40940ddca458db0f091c07f19b740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:06.778235 19814 raft_consensus.cc:1270] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:06.779282 19999 consensus_queue.cc:1035] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:06.788309 19740 raft_consensus.cc:1270] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:06.790014 19985 consensus_queue.cc:1035] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:06.827153 19814 raft_consensus.cc:1270] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:06.828164 19976 consensus_queue.cc:1035] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:06.835147 19664 raft_consensus.cc:1270] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:06.836517 19976 consensus_queue.cc:1035] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:06.986253 19740 raft_consensus.cc:1270] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:06.987387 19983 consensus_queue.cc:1035] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:06.994172 19664 raft_consensus.cc:1270] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:06.996083 19861 consensus_queue.cc:1035] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:07.012394 19814 raft_consensus.cc:1270] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:07.013465 19976 consensus_queue.cc:1035] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:07.020036 19664 raft_consensus.cc:1270] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:07.023341 20008 consensus_queue.cc:1035] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:07.063297 19814 raft_consensus.cc:1270] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:07.064359 19941 consensus_queue.cc:1035] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:07.067601 19664 raft_consensus.cc:1270] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:07.068792 20008 consensus_queue.cc:1035] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:07.074294 19740 raft_consensus.cc:1270] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Refusing update from remote peer f0b8ad29373745ddac1bcb23ca06cf25: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:07.075747 19985 consensus_queue.cc:1035] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:07.082827 19814 raft_consensus.cc:1270] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:07.084378 19976 consensus_queue.cc:1035] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:07.160410 19814 raft_consensus.cc:1270] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:07.161772 19877 consensus_queue.cc:1035] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:07.168792 19664 raft_consensus.cc:1270] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Refusing update from remote peer f9d40940ddca458db0f091c07f19b740: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:07.170192 19976 consensus_queue.cc:1035] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:09.284093 19560 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58578:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20250112 20:59:09.301793 19560 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:58578:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20250112 20:59:09.342218 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
W20250112 20:59:09.532845 20052 log.cc:926] Time spent T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: Append to log took a long time: real 0.059s user 0.001s sys 0.000s
W20250112 20:59:09.602058 19628 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:42611, user_credentials={real_user=slave}} blocked reactor thread for 64151.9us
I20250112 20:59:09.668884 20071 tablet.cc:1721] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.678084 20076 tablet.cc:1721] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.743942 20082 tablet.cc:1721] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.744848 19663 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
mode: GRACEFUL
new_leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:38942
I20250112 20:59:09.745548 19663 raft_consensus.cc:604] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Received request to transfer leadership to TS b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:09.750217 20081 tablet.cc:1721] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250112 20:59:09.758983 19628 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.130:42611, user_credentials={real_user=slave}} blocked reactor thread for 48543.8us
I20250112 20:59:09.772832 20083 tablet.cc:1721] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.714582 20078 tablet.cc:1721] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250112 20:59:09.791685 20085 log.cc:926] Time spent T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20250112 20:59:09.771451 20086 tablet.cc:1721] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250112 20:59:09.841133 19706 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.129:34089, user_credentials={real_user=slave}} blocked reactor thread for 90201.3us
I20250112 20:59:09.844477 20076 tablet.cc:1721] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.851454 20081 tablet.cc:1721] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.870391 20097 tablet.cc:1721] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.871093 20099 tablet.cc:1721] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.890380 20089 raft_consensus.cc:988] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25: : Instructing follower b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:09.890901 19982 raft_consensus.cc:1076] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Signalling peer b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:09.894130 20099 tablet.cc:1721] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.895763 20078 tablet.cc:1721] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.901063 20071 tablet.cc:1721] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.912899 20082 tablet.cc:1721] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.913087 20083 tablet.cc:1721] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.903060 20076 tablet.cc:1721] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.921379 19808 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:57334
I20250112 20:59:09.923242 19808 raft_consensus.cc:491] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:09.925071 19808 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:09.924239 20086 tablet.cc:1721] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.941233 19808 raft_consensus.cc:513] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:09.943279 19808 leader_election.cc:290] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 3 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:09.944084 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:09.944681 19739 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:09.961395 20081 tablet.cc:1721] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.964228 19739 raft_consensus.cc:2463] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 3.
I20250112 20:59:09.965485 20100 tablet.cc:1721] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.965966 20096 tablet.cc:1721] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.965550 19777 leader_election.cc:304] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f9d40940ddca458db0f091c07f19b740; no voters:
I20250112 20:59:09.968189 20068 raft_consensus.cc:2798] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:59:09.968788 20068 raft_consensus.cc:695] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:09.969024 20098 tablet.cc:1721] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.969573 20068 consensus_queue.cc:237] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:09.975694 20076 tablet.cc:1721] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.988663 20098 tablet.cc:1721] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.993256 20081 tablet.cc:1721] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.990500 19560 catalog_manager.cc:5526] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 2 to 3, leader changed from f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129) to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 3 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:09.994711 20083 tablet.cc:1721] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:09.989286 20096 tablet.cc:1721] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:10.003103 19661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "395b5e250e9e40cdb1ba27ee7dca3cc7" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:10.003693 19661 raft_consensus.cc:3049] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:59:10.003988 19661 raft_consensus.cc:738] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:10.004804 19661 consensus_queue.cc:260] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.006047 19661 raft_consensus.cc:3054] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.042861 19739 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
mode: GRACEFUL
new_leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:58590
I20250112 20:59:10.043442 19739 raft_consensus.cc:604] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Received request to transfer leadership to TS b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:10.044647 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20250112 20:59:10.045382 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:10.046259 19661 raft_consensus.cc:2463] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 3.
I20250112 20:59:10.068011 19661 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6"
dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
mode: GRACEFUL
new_leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:38942
I20250112 20:59:10.068655 19661 raft_consensus.cc:604] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Received request to transfer leadership to TS b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:10.068876 19562 catalog_manager.cc:6071] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f: soft_deleted_table [id=0be7045386c74ecf8bbea7f15f589bb2] alter complete (version 1)
I20250112 20:59:10.070956 19739 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9"
dest_uuid: "f9d40940ddca458db0f091c07f19b740"
mode: GRACEFUL
new_leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:58590
I20250112 20:59:10.071471 19739 raft_consensus.cc:604] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Received request to transfer leadership to TS b24e8e78a6834a6fae06c193b5ac43a9
I20250112 20:59:10.072482 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:59:10.073055 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:10.079764 20066 raft_consensus.cc:988] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25: : Instructing follower b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:10.080236 20066 raft_consensus.cc:1076] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Signalling peer b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:10.082295 19808 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:57334
I20250112 20:59:10.082846 19808 raft_consensus.cc:491] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:10.083204 19808 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.090655 19877 raft_consensus.cc:988] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740: : Instructing follower b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:10.091130 19877 raft_consensus.cc:1076] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Signalling peer b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:10.091073 19808 raft_consensus.cc:513] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.092654 19813 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:57350
I20250112 20:59:10.093114 19813 raft_consensus.cc:491] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:10.093161 19808 leader_election.cc:290] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 3 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:10.093497 19813 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.095106 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:10.095503 19661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc5e3870dde48d1a07112c7c2a295a6" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:10.095841 19739 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.096233 19661 raft_consensus.cc:3049] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:59:10.096562 19661 raft_consensus.cc:738] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f0b8ad29373745ddac1bcb23ca06cf25, State: Running, Role: LEADER
I20250112 20:59:10.097302 19661 consensus_queue.cc:260] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.098433 19661 raft_consensus.cc:3054] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.100085 19813 raft_consensus.cc:513] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.102068 19813 leader_election.cc:290] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 3 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:10.102509 19739 raft_consensus.cc:2463] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 3.
I20250112 20:59:10.103314 19664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:10.103958 19664 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.104010 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d69c5861f39b4aa0b8396f278fb22bc9" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:10.105293 19739 raft_consensus.cc:3049] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:59:10.105747 19739 raft_consensus.cc:738] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:10.106922 19661 raft_consensus.cc:2463] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 3.
I20250112 20:59:10.106757 19739 consensus_queue.cc:260] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.108032 19780 leader_election.cc:304] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:10.108142 19739 raft_consensus.cc:3054] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.108907 20068 raft_consensus.cc:2798] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:59:10.109424 20068 raft_consensus.cc:695] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:10.109850 19664 raft_consensus.cc:2463] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 3.
I20250112 20:59:10.110687 19780 leader_election.cc:304] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:10.110221 20068 consensus_queue.cc:237] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.111430 20072 raft_consensus.cc:2798] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:59:10.111876 20072 raft_consensus.cc:695] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:10.112779 20072 consensus_queue.cc:237] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.118703 19739 raft_consensus.cc:2463] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 3.
I20250112 20:59:10.128002 19562 catalog_manager.cc:5526] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 2 to 3, leader changed from f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129) to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 3 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:10.139801 19562 catalog_manager.cc:5526] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 2 to 3, leader changed from f9d40940ddca458db0f091c07f19b740 (127.15.23.130) to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 3 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:10.387748 19661 raft_consensus.cc:1270] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250112 20:59:10.389041 20068 consensus_queue.cc:1035] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:59:10.398000 19739 raft_consensus.cc:1270] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250112 20:59:10.399405 20028 consensus_queue.cc:1035] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20250112 20:59:10.513975 19739 raft_consensus.cc:1270] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:59:10.515040 20108 consensus_queue.cc:1035] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:59:10.523944 19661 raft_consensus.cc:1270] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:59:10.525126 20108 consensus_queue.cc:1035] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:59:10.531961 20062 raft_consensus.cc:988] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: : Instructing follower b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:10.532454 19877 raft_consensus.cc:1076] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Signalling peer b24e8e78a6834a6fae06c193b5ac43a9 to start an election
I20250112 20:59:10.533941 19813 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093"
dest_uuid: "b24e8e78a6834a6fae06c193b5ac43a9"
from {username='slave'} at 127.0.0.1:57350
I20250112 20:59:10.534449 19813 raft_consensus.cc:491] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:10.534798 19813 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.540376 19813 raft_consensus.cc:513] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.542054 19813 leader_election.cc:290] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 3 election: Requested vote from peers f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089), f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611)
I20250112 20:59:10.542461 19661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f0b8ad29373745ddac1bcb23ca06cf25"
I20250112 20:59:10.543114 19661 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.543339 19739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffbb7ca86d9745bf8365c49b9e629093" candidate_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f9d40940ddca458db0f091c07f19b740"
I20250112 20:59:10.544010 19739 raft_consensus.cc:3049] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:59:10.544313 19739 raft_consensus.cc:738] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9d40940ddca458db0f091c07f19b740, State: Running, Role: LEADER
I20250112 20:59:10.545010 19739 consensus_queue.cc:260] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.545899 19739 raft_consensus.cc:3054] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:10.548499 19740 raft_consensus.cc:1270] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:59:10.548738 19661 raft_consensus.cc:2463] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 3.
I20250112 20:59:10.549870 19780 leader_election.cc:304] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24e8e78a6834a6fae06c193b5ac43a9, f0b8ad29373745ddac1bcb23ca06cf25; no voters:
I20250112 20:59:10.550331 20108 consensus_queue.cc:1035] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:59:10.550603 20072 raft_consensus.cc:2798] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:59:10.551144 20072 raft_consensus.cc:695] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 LEADER]: Becoming Leader. State: Replica: b24e8e78a6834a6fae06c193b5ac43a9, State: Running, Role: LEADER
I20250112 20:59:10.551436 19739 raft_consensus.cc:2463] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24e8e78a6834a6fae06c193b5ac43a9 in term 3.
I20250112 20:59:10.552440 20072 consensus_queue.cc:237] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } }
I20250112 20:59:10.560506 19661 raft_consensus.cc:1270] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:59:10.561662 19563 catalog_manager.cc:5526] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 reported cstate change: term changed from 2 to 3, leader changed from f9d40940ddca458db0f091c07f19b740 (127.15.23.130) to b24e8e78a6834a6fae06c193b5ac43a9 (127.15.23.131). New cstate: current_term: 3 leader_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24e8e78a6834a6fae06c193b5ac43a9" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 41093 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:10.562047 20028 consensus_queue.cc:1035] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:59:10.931434 19664 raft_consensus.cc:1270] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250112 20:59:10.932456 20028 consensus_queue.cc:1035] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0b8ad29373745ddac1bcb23ca06cf25" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:59:10.940280 19739 raft_consensus.cc:1270] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Refusing update from remote peer b24e8e78a6834a6fae06c193b5ac43a9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250112 20:59:10.941291 20072 consensus_queue.cc:1035] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d40940ddca458db0f091c07f19b740" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250112 20:59:11.073771 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:11.079771 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:11.080160 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:11.085037 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:11.085333 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:12.086094 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:12.091393 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:12.091846 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:12.096114 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:12.096469 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:13.097208 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:13.101802 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:13.102178 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:13.106354 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:13.106705 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:14.107949 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:14.116464 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:14.116938 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:14.123071 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:14.123504 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:15.458817 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:15.463271 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:15.463629 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:15.467748 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:15.468075 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20250112 20:59:15.678508 19594 debug-util.cc:398] Leaking SignalData structure 0x7b08000e6a60 after lost signal to thread 15457
W20250112 20:59:15.679486 19594 debug-util.cc:398] Leaking SignalData structure 0x7b08002f51e0 after lost signal to thread 19597
W20250112 20:59:15.680409 19594 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1fe0 after lost signal to thread 19689
W20250112 20:59:15.681281 19594 debug-util.cc:398] Leaking SignalData structure 0x7b0800296180 after lost signal to thread 19764
W20250112 20:59:15.682303 19594 debug-util.cc:398] Leaking SignalData structure 0x7b08002d0da0 after lost signal to thread 19838
I20250112 20:59:16.469053 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:16.476704 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:16.477247 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:16.484440 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:16.484989 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:17.485949 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:17.493705 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:17.494226 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:17.501657 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:17.502141 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:18.503036 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:18.508978 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:18.509330 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:18.513445 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:18.513753 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:19.514496 15454 auto_leader_rebalancer.cc:140] leader rebalance for table soft_deleted_table
I20250112 20:59:19.520100 15454 auto_leader_rebalancer.cc:388] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20250112 20:59:19.520556 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:19.525406 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:19.525696 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:20.573457 15454 tablet_server.cc:178] TabletServer@127.15.23.129:0 shutting down...
I20250112 20:59:20.600412 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:59:20.601155 15454 tablet_replica.cc:331] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.601917 15454 raft_consensus.cc:2238] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.602545 15454 raft_consensus.cc:2267] T bbb4975e81294cf8a61f7c22304517f9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.604892 15454 tablet_replica.cc:331] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.605490 15454 raft_consensus.cc:2238] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.606009 15454 raft_consensus.cc:2267] T da5fcd0116034ac5939b328651511d10 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.608433 15454 tablet_replica.cc:331] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.609138 15454 raft_consensus.cc:2238] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.610206 15454 raft_consensus.cc:2267] T ffae6c92a63042b7b52e2c8c70d91648 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.612581 15454 tablet_replica.cc:331] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.613161 15454 raft_consensus.cc:2238] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.613938 15454 raft_consensus.cc:2267] T 55452388fe9a4cb38297efb9e7e0e54e P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.616490 15454 tablet_replica.cc:331] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.617120 15454 raft_consensus.cc:2238] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.617959 15454 raft_consensus.cc:2267] T 3226df3d47f748b28b4dcce8ea2f36e9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.619915 15454 tablet_replica.cc:331] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.620389 15454 raft_consensus.cc:2238] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.620779 15454 raft_consensus.cc:2267] T 4fc5e3870dde48d1a07112c7c2a295a6 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.622655 15454 tablet_replica.cc:331] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.623153 15454 raft_consensus.cc:2238] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.623932 15454 raft_consensus.cc:2267] T 6bb774316f894a44984da5099d1df61c P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.625886 15454 tablet_replica.cc:331] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.626400 15454 raft_consensus.cc:2238] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.626907 15454 raft_consensus.cc:2267] T ffbb7ca86d9745bf8365c49b9e629093 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.629128 15454 tablet_replica.cc:331] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.629618 15454 raft_consensus.cc:2238] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.630016 15454 raft_consensus.cc:2267] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.631842 15454 tablet_replica.cc:331] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.632391 15454 raft_consensus.cc:2238] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.632799 15454 raft_consensus.cc:2267] T 68785fca3fe943b49359b52a5545a970 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.634711 15454 tablet_replica.cc:331] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.635229 15454 raft_consensus.cc:2238] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.635658 15454 raft_consensus.cc:2267] T 9cb6313eae3d4cbd9526bde23d688b51 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.637450 15454 tablet_replica.cc:331] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.637938 15454 raft_consensus.cc:2238] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.638654 15454 raft_consensus.cc:2267] T e38073eed2194f91bbed0bba42455734 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.640656 15454 tablet_replica.cc:331] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.641103 15454 raft_consensus.cc:2238] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.641533 15454 raft_consensus.cc:2267] T 861abe0f055a430a950e7c7573db1947 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.643534 15454 tablet_replica.cc:331] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.644024 15454 raft_consensus.cc:2238] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.644781 15454 raft_consensus.cc:2267] T 1ee909bd13c44a2c85185e4ae376cab5 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.646656 15454 tablet_replica.cc:331] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.647221 15454 raft_consensus.cc:2238] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.647889 15454 raft_consensus.cc:2267] T 179f24ca8e194189a8940dafe1c65418 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:20.647923 19780 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 78 similar messages]
I20250112 20:59:20.650713 15454 tablet_replica.cc:331] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
W20250112 20:59:20.651175 19780 consensus_peers.cc:487] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 -> Peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Couldn't send request to peer f0b8ad29373745ddac1bcb23ca06cf25. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:20.651309 15454 raft_consensus.cc:2238] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.651785 15454 raft_consensus.cc:2267] T 6c58a59765104b37b0cbc48a3a6a2902 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.653760 15454 tablet_replica.cc:331] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.654215 15454 raft_consensus.cc:2238] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.654675 15454 raft_consensus.cc:2267] T d69c5861f39b4aa0b8396f278fb22bc9 P f0b8ad29373745ddac1bcb23ca06cf25 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.656500 15454 tablet_replica.cc:331] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25: stopping tablet replica
I20250112 20:59:20.656983 15454 raft_consensus.cc:2238] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.657457 15454 raft_consensus.cc:2267] T 30a1811da1844978b18649248a5a3f20 P f0b8ad29373745ddac1bcb23ca06cf25 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.699337 15454 tablet_server.cc:195] TabletServer@127.15.23.129:0 shutdown complete.
I20250112 20:59:20.736027 15454 tablet_server.cc:178] TabletServer@127.15.23.130:0 shutting down...
W20250112 20:59:20.736884 19706 consensus_peers.cc:487] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 -> Peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Couldn't send request to peer f0b8ad29373745ddac1bcb23ca06cf25. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:34089: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:59:20.743443 19706 consensus_peers.cc:487] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 -> Peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Couldn't send request to peer f0b8ad29373745ddac1bcb23ca06cf25. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:34089: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:20.763020 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:59:20.763626 15454 tablet_replica.cc:331] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.764254 15454 raft_consensus.cc:2238] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.765136 15454 raft_consensus.cc:2267] T bbb4975e81294cf8a61f7c22304517f9 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.767028 15454 tablet_replica.cc:331] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.767571 15454 raft_consensus.cc:2238] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.768319 15454 raft_consensus.cc:2267] T da5fcd0116034ac5939b328651511d10 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.770337 15454 tablet_replica.cc:331] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.770958 15454 raft_consensus.cc:2238] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.771413 15454 raft_consensus.cc:2267] T ffbb7ca86d9745bf8365c49b9e629093 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.773258 15454 tablet_replica.cc:331] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.773746 15454 raft_consensus.cc:2238] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.774201 15454 raft_consensus.cc:2267] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.776166 15454 tablet_replica.cc:331] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.776712 15454 raft_consensus.cc:2238] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.777154 15454 raft_consensus.cc:2267] T ffae6c92a63042b7b52e2c8c70d91648 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.779104 15454 tablet_replica.cc:331] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.779618 15454 raft_consensus.cc:2238] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.780087 15454 raft_consensus.cc:2267] T 55452388fe9a4cb38297efb9e7e0e54e P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.782020 15454 tablet_replica.cc:331] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.782473 15454 raft_consensus.cc:2238] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.782894 15454 raft_consensus.cc:2267] T 3226df3d47f748b28b4dcce8ea2f36e9 P f9d40940ddca458db0f091c07f19b740 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.784689 15454 tablet_replica.cc:331] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.785166 15454 raft_consensus.cc:2238] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.785610 15454 raft_consensus.cc:2267] T d69c5861f39b4aa0b8396f278fb22bc9 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.787698 15454 tablet_replica.cc:331] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.788211 15454 raft_consensus.cc:2238] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.789355 15454 raft_consensus.cc:2267] T 9cb6313eae3d4cbd9526bde23d688b51 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.791527 15454 tablet_replica.cc:331] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.792052 15454 raft_consensus.cc:2238] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.792582 15454 raft_consensus.cc:2267] T 68785fca3fe943b49359b52a5545a970 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.794477 15454 tablet_replica.cc:331] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.794947 15454 raft_consensus.cc:2238] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.795655 15454 raft_consensus.cc:2267] T 30a1811da1844978b18649248a5a3f20 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.797518 15454 tablet_replica.cc:331] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.798012 15454 raft_consensus.cc:2238] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.798405 15454 raft_consensus.cc:2267] T 6bb774316f894a44984da5099d1df61c P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.800103 15454 tablet_replica.cc:331] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.800549 15454 raft_consensus.cc:2238] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.801016 15454 raft_consensus.cc:2267] T 4fc5e3870dde48d1a07112c7c2a295a6 P f9d40940ddca458db0f091c07f19b740 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.802803 15454 tablet_replica.cc:331] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.803282 15454 raft_consensus.cc:2238] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.803719 15454 raft_consensus.cc:2267] T 179f24ca8e194189a8940dafe1c65418 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.805855 15454 tablet_replica.cc:331] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.806334 15454 raft_consensus.cc:2238] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.807230 15454 raft_consensus.cc:2267] T 6c58a59765104b37b0cbc48a3a6a2902 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:20.807379 19777 consensus_peers.cc:487] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 -> Peer f9d40940ddca458db0f091c07f19b740 (127.15.23.130:42611): Couldn't send request to peer f9d40940ddca458db0f091c07f19b740. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:20.809208 15454 tablet_replica.cc:331] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.809800 15454 raft_consensus.cc:2238] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.810679 15454 raft_consensus.cc:2267] T 861abe0f055a430a950e7c7573db1947 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.812950 15454 tablet_replica.cc:331] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.813413 15454 raft_consensus.cc:2238] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.813887 15454 raft_consensus.cc:2267] T 1ee909bd13c44a2c85185e4ae376cab5 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:20.814306 19780 consensus_peers.cc:487] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 -> Peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Couldn't send request to peer f0b8ad29373745ddac1bcb23ca06cf25. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:34089: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:20.815726 15454 tablet_replica.cc:331] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740: stopping tablet replica
I20250112 20:59:20.816255 15454 raft_consensus.cc:2238] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.816728 15454 raft_consensus.cc:2267] T e38073eed2194f91bbed0bba42455734 P f9d40940ddca458db0f091c07f19b740 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:20.830840 19780 consensus_peers.cc:487] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 -> Peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Couldn't send request to peer f0b8ad29373745ddac1bcb23ca06cf25. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:34089: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:20.854782 15454 tablet_server.cc:195] TabletServer@127.15.23.130:0 shutdown complete.
I20250112 20:59:20.887632 15454 tablet_server.cc:178] TabletServer@127.15.23.131:0 shutting down...
I20250112 20:59:20.913353 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:59:20.914026 15454 tablet_replica.cc:331] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.914896 15454 raft_consensus.cc:2238] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.915524 15454 raft_consensus.cc:2267] T da5fcd0116034ac5939b328651511d10 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.917502 15454 tablet_replica.cc:331] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.917986 15454 raft_consensus.cc:2238] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.918375 15454 raft_consensus.cc:2267] T bbb4975e81294cf8a61f7c22304517f9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.920140 15454 tablet_replica.cc:331] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.920668 15454 raft_consensus.cc:2238] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.921082 15454 raft_consensus.cc:2267] T ffae6c92a63042b7b52e2c8c70d91648 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.922699 15454 tablet_replica.cc:331] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.923123 15454 raft_consensus.cc:2238] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.923534 15454 raft_consensus.cc:2267] T 55452388fe9a4cb38297efb9e7e0e54e P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.925220 15454 tablet_replica.cc:331] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.925637 15454 raft_consensus.cc:2238] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.925971 15454 raft_consensus.cc:2267] T 3226df3d47f748b28b4dcce8ea2f36e9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.927649 15454 tablet_replica.cc:331] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.928092 15454 raft_consensus.cc:2238] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.928841 15454 raft_consensus.cc:2267] T 4fc5e3870dde48d1a07112c7c2a295a6 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.930522 15454 tablet_replica.cc:331] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.930939 15454 raft_consensus.cc:2238] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.931285 15454 raft_consensus.cc:2267] T 6bb774316f894a44984da5099d1df61c P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.932988 15454 tablet_replica.cc:331] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.933430 15454 raft_consensus.cc:2238] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.933755 15454 raft_consensus.cc:2267] T 9cb6313eae3d4cbd9526bde23d688b51 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.935295 15454 tablet_replica.cc:331] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.935703 15454 raft_consensus.cc:2238] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.936358 15454 raft_consensus.cc:2267] T 68785fca3fe943b49359b52a5545a970 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.938009 15454 tablet_replica.cc:331] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.938467 15454 raft_consensus.cc:2238] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.939109 15454 raft_consensus.cc:2267] T d69c5861f39b4aa0b8396f278fb22bc9 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.940778 15454 tablet_replica.cc:331] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.941216 15454 raft_consensus.cc:2238] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.941545 15454 raft_consensus.cc:2267] T 30a1811da1844978b18649248a5a3f20 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.943498 15454 tablet_replica.cc:331] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.944000 15454 raft_consensus.cc:2238] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.944914 15454 raft_consensus.cc:2267] T 179f24ca8e194189a8940dafe1c65418 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:20.945881 19780 consensus_peers.cc:487] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 -> Peer f0b8ad29373745ddac1bcb23ca06cf25 (127.15.23.129:34089): Couldn't send request to peer f0b8ad29373745ddac1bcb23ca06cf25. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:34089: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:20.947000 15454 tablet_replica.cc:331] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.947511 15454 raft_consensus.cc:2238] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.947861 15454 raft_consensus.cc:2267] T 6c58a59765104b37b0cbc48a3a6a2902 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.949549 15454 tablet_replica.cc:331] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.949955 15454 raft_consensus.cc:2238] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.950318 15454 raft_consensus.cc:2267] T 861abe0f055a430a950e7c7573db1947 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.952023 15454 tablet_replica.cc:331] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.952448 15454 raft_consensus.cc:2238] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.952841 15454 raft_consensus.cc:2267] T 1ee909bd13c44a2c85185e4ae376cab5 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.954407 15454 tablet_replica.cc:331] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.954790 15454 raft_consensus.cc:2238] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.955421 15454 raft_consensus.cc:2267] T ffbb7ca86d9745bf8365c49b9e629093 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.957088 15454 tablet_replica.cc:331] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.957638 15454 raft_consensus.cc:2238] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:59:20.958374 15454 raft_consensus.cc:2267] T 395b5e250e9e40cdb1ba27ee7dca3cc7 P b24e8e78a6834a6fae06c193b5ac43a9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.960022 15454 tablet_replica.cc:331] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9: stopping tablet replica
I20250112 20:59:20.960381 15454 raft_consensus.cc:2238] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:20.960762 15454 raft_consensus.cc:2267] T e38073eed2194f91bbed0bba42455734 P b24e8e78a6834a6fae06c193b5ac43a9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:20.997582 15454 tablet_server.cc:195] TabletServer@127.15.23.131:0 shutdown complete.
I20250112 20:59:21.026687 15454 master.cc:537] Master@127.15.23.190:45463 shutting down...
I20250112 20:59:21.046458 15454 raft_consensus.cc:2238] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:21.046967 15454 raft_consensus.cc:2267] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:21.047272 15454 tablet_replica.cc:331] T 00000000000000000000000000000000 P ce75ee801a7543fa95ca9d8ad03ca27f: stopping tablet replica
I20250112 20:59:21.081475 15454 master.cc:559] Master@127.15.23.190:45463 shutdown complete.
[ OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (20669 ms)
[ RUN ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20250112 20:59:21.129845 15454 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.23.190:45649
I20250112 20:59:21.130831 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:59:21.136173 20146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:21.137001 20147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:21.137871 20149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:21.138422 15454 server_base.cc:1034] running on GCE node
I20250112 20:59:21.139300 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:59:21.139485 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:59:21.139614 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715561139602 us; error 0 us; skew 500 ppm
I20250112 20:59:21.140012 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:59:21.145306 15454 webserver.cc:458] Webserver started at http://127.15.23.190:34137/ using document root <none> and password file <none>
I20250112 20:59:21.145720 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:59:21.145869 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:59:21.146067 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:59:21.147123 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/master-0-root/instance:
uuid: "e3919277e93d4feebc3fd09cbc3810e6"
format_stamp: "Formatted at 2025-01-12 20:59:21 on dist-test-slave-m2nm"
I20250112 20:59:21.151221 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250112 20:59:21.154074 20154 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.154726 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250112 20:59:21.154973 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/master-0-root
uuid: "e3919277e93d4feebc3fd09cbc3810e6"
format_stamp: "Formatted at 2025-01-12 20:59:21 on dist-test-slave-m2nm"
I20250112 20:59:21.155210 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:59:21.166982 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:59:21.168049 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:59:21.201900 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.190:45649
I20250112 20:59:21.201982 20205 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.190:45649 every 8 connection(s)
I20250112 20:59:21.205312 20206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.215201 20206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6: Bootstrap starting.
I20250112 20:59:21.219425 20206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:21.222870 20206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6: No bootstrap required, opened a new log
I20250112 20:59:21.224622 20206 raft_consensus.cc:357] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3919277e93d4feebc3fd09cbc3810e6" member_type: VOTER }
I20250112 20:59:21.225057 20206 raft_consensus.cc:383] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:21.225271 20206 raft_consensus.cc:738] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3919277e93d4feebc3fd09cbc3810e6, State: Initialized, Role: FOLLOWER
I20250112 20:59:21.225755 20206 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3919277e93d4feebc3fd09cbc3810e6" member_type: VOTER }
I20250112 20:59:21.226146 20206 raft_consensus.cc:397] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250112 20:59:21.226366 20206 raft_consensus.cc:491] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250112 20:59:21.226612 20206 raft_consensus.cc:3054] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:21.230368 20206 raft_consensus.cc:513] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3919277e93d4feebc3fd09cbc3810e6" member_type: VOTER }
I20250112 20:59:21.230803 20206 leader_election.cc:304] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3919277e93d4feebc3fd09cbc3810e6; no voters:
I20250112 20:59:21.231822 20206 leader_election.cc:290] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250112 20:59:21.232180 20209 raft_consensus.cc:2798] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:21.233498 20209 raft_consensus.cc:695] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 1 LEADER]: Becoming Leader. State: Replica: e3919277e93d4feebc3fd09cbc3810e6, State: Running, Role: LEADER
I20250112 20:59:21.234076 20209 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3919277e93d4feebc3fd09cbc3810e6" member_type: VOTER }
I20250112 20:59:21.234632 20206 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [sys.catalog]: configured and running, proceeding with master startup.
I20250112 20:59:21.239142 20210 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3919277e93d4feebc3fd09cbc3810e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3919277e93d4feebc3fd09cbc3810e6" member_type: VOTER } }
I20250112 20:59:21.239564 20211 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3919277e93d4feebc3fd09cbc3810e6. Latest consensus state: current_term: 1 leader_uuid: "e3919277e93d4feebc3fd09cbc3810e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3919277e93d4feebc3fd09cbc3810e6" member_type: VOTER } }
I20250112 20:59:21.239919 20210 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [sys.catalog]: This master's current role is: LEADER
I20250112 20:59:21.240301 20211 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [sys.catalog]: This master's current role is: LEADER
I20250112 20:59:21.241851 20217 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250112 20:59:21.246131 20217 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250112 20:59:21.248687 15454 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250112 20:59:21.253509 20217 catalog_manager.cc:1348] Generated new cluster ID: 1e82f17c77fe4851a92729723c4d41e7
I20250112 20:59:21.253742 20217 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250112 20:59:21.274482 20217 catalog_manager.cc:1371] Generated new certificate authority record
I20250112 20:59:21.275672 20217 catalog_manager.cc:1505] Loading token signing keys...
I20250112 20:59:21.287122 20217 catalog_manager.cc:5899] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6: Generated new TSK 0
I20250112 20:59:21.287679 20217 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250112 20:59:21.315136 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:59:21.320842 20227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:21.322166 20228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:21.323441 20230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:21.324648 15454 server_base.cc:1034] running on GCE node
I20250112 20:59:21.325428 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:59:21.325587 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:59:21.325721 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715561325710 us; error 0 us; skew 500 ppm
I20250112 20:59:21.326128 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:59:21.328130 15454 webserver.cc:458] Webserver started at http://127.15.23.129:41489/ using document root <none> and password file <none>
I20250112 20:59:21.328547 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:59:21.328768 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:59:21.328971 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:59:21.329908 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-0-root/instance:
uuid: "792c9d15fc44406b9058909c5a860f96"
format_stamp: "Formatted at 2025-01-12 20:59:21 on dist-test-slave-m2nm"
I20250112 20:59:21.333642 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250112 20:59:21.336534 20235 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.337261 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250112 20:59:21.337504 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-0-root
uuid: "792c9d15fc44406b9058909c5a860f96"
format_stamp: "Formatted at 2025-01-12 20:59:21 on dist-test-slave-m2nm"
I20250112 20:59:21.337723 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:59:21.358767 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:59:21.359730 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:59:21.360931 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:59:21.363075 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:59:21.363236 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.363476 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:59:21.363610 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.400588 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.129:39087
I20250112 20:59:21.400718 20297 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.129:39087 every 8 connection(s)
I20250112 20:59:21.404927 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:59:21.412386 20302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:21.413336 20303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:21.415725 15454 server_base.cc:1034] running on GCE node
W20250112 20:59:21.416764 20305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:21.417307 20298 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45649
I20250112 20:59:21.417690 20298 heartbeater.cc:463] Registering TS with master...
I20250112 20:59:21.417769 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:59:21.418037 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:59:21.418188 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715561418168 us; error 0 us; skew 500 ppm
I20250112 20:59:21.418383 20298 heartbeater.cc:510] Master 127.15.23.190:45649 requested a full tablet report, sending...
I20250112 20:59:21.418782 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:59:21.420192 20171 ts_manager.cc:194] Registered new tserver with Master: 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087)
I20250112 20:59:21.421581 15454 webserver.cc:458] Webserver started at http://127.15.23.130:33013/ using document root <none> and password file <none>
I20250112 20:59:21.422067 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:59:21.422044 20171 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33324
I20250112 20:59:21.422417 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:59:21.422722 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:59:21.423933 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-1-root/instance:
uuid: "2a2063dd1fee4cb48fe32bf917055142"
format_stamp: "Formatted at 2025-01-12 20:59:21 on dist-test-slave-m2nm"
I20250112 20:59:21.427990 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250112 20:59:21.431111 20310 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.431721 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250112 20:59:21.431983 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-1-root
uuid: "2a2063dd1fee4cb48fe32bf917055142"
format_stamp: "Formatted at 2025-01-12 20:59:21 on dist-test-slave-m2nm"
I20250112 20:59:21.432233 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:59:21.447919 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:59:21.449044 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:59:21.450335 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:59:21.452306 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:59:21.452478 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.452697 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:59:21.452843 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.488293 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.130:36853
I20250112 20:59:21.488389 20372 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.130:36853 every 8 connection(s)
I20250112 20:59:21.492673 15454 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250112 20:59:21.499845 20376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250112 20:59:21.501144 20377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:21.505136 20373 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45649
I20250112 20:59:21.505589 20373 heartbeater.cc:463] Registering TS with master...
I20250112 20:59:21.506273 20373 heartbeater.cc:510] Master 127.15.23.190:45649 requested a full tablet report, sending...
W20250112 20:59:21.506351 20379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250112 20:59:21.507434 15454 server_base.cc:1034] running on GCE node
I20250112 20:59:21.508126 20171 ts_manager.cc:194] Registered new tserver with Master: 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:21.508245 15454 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250112 20:59:21.508565 15454 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250112 20:59:21.508751 15454 hybrid_clock.cc:648] HybridClock initialized: now 1736715561508731 us; error 0 us; skew 500 ppm
I20250112 20:59:21.509294 15454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250112 20:59:21.509657 20171 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33334
I20250112 20:59:21.511735 15454 webserver.cc:458] Webserver started at http://127.15.23.131:42365/ using document root <none> and password file <none>
I20250112 20:59:21.512228 15454 fs_manager.cc:362] Metadata directory not provided
I20250112 20:59:21.512394 15454 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250112 20:59:21.512660 15454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250112 20:59:21.513720 15454 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-2-root/instance:
uuid: "0c62e542bb824359a6a7a83070a62b61"
format_stamp: "Formatted at 2025-01-12 20:59:21 on dist-test-slave-m2nm"
I20250112 20:59:21.517904 15454 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20250112 20:59:21.520797 20384 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.521405 15454 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250112 20:59:21.521647 15454 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-2-root
uuid: "0c62e542bb824359a6a7a83070a62b61"
format_stamp: "Formatted at 2025-01-12 20:59:21 on dist-test-slave-m2nm"
I20250112 20:59:21.521919 15454 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250112 20:59:21.535554 15454 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250112 20:59:21.536572 15454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250112 20:59:21.537853 15454 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250112 20:59:21.539816 15454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250112 20:59:21.540006 15454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.540222 15454 ts_tablet_manager.cc:610] Registered 0 tablets
I20250112 20:59:21.540385 15454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250112 20:59:21.577445 15454 rpc_server.cc:307] RPC server started. Bound to: 127.15.23.131:46265
I20250112 20:59:21.577569 20446 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.23.131:46265 every 8 connection(s)
I20250112 20:59:21.590209 20447 heartbeater.cc:346] Connected to a master server at 127.15.23.190:45649
I20250112 20:59:21.590608 20447 heartbeater.cc:463] Registering TS with master...
I20250112 20:59:21.591198 20447 heartbeater.cc:510] Master 127.15.23.190:45649 requested a full tablet report, sending...
I20250112 20:59:21.592872 20171 ts_manager.cc:194] Registered new tserver with Master: 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:21.593034 15454 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.012213641s
I20250112 20:59:21.594604 20171 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33338
I20250112 20:59:21.595870 15454 test_util.cc:274] Using random seed: -1709182006
I20250112 20:59:21.615974 20171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33342:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250112 20:59:21.618618 20171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250112 20:59:21.690227 20338 tablet_service.cc:1467] Processing CreateTablet for tablet 13960ac9c4094cf7b7a843d3a768433b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:21.690817 20263 tablet_service.cc:1467] Processing CreateTablet for tablet 13960ac9c4094cf7b7a843d3a768433b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:21.691569 20338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13960ac9c4094cf7b7a843d3a768433b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.692133 20263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13960ac9c4094cf7b7a843d3a768433b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.693363 20260 tablet_service.cc:1467] Processing CreateTablet for tablet 32907ca5566a44f9a3f284ba36e744ae (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:21.694602 20260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32907ca5566a44f9a3f284ba36e744ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.694828 20256 tablet_service.cc:1467] Processing CreateTablet for tablet 43aad0e784aa4014b483577c8564bea4 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:21.696045 20256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43aad0e784aa4014b483577c8564bea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.697484 20337 tablet_service.cc:1467] Processing CreateTablet for tablet c9181f2ac504418faf8c6291311cd19b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:21.698516 20337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9181f2ac504418faf8c6291311cd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.701759 20257 tablet_service.cc:1467] Processing CreateTablet for tablet 42b0e5bc9e0749c7b060f68f8855a77b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:21.702996 20257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42b0e5bc9e0749c7b060f68f8855a77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.705713 20258 tablet_service.cc:1467] Processing CreateTablet for tablet c59266f7526c46dc9dadcb79c5e3981a (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:21.706099 20255 tablet_service.cc:1467] Processing CreateTablet for tablet c439b8ab09aa4503a93dc9f98400a681 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:21.706924 20258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59266f7526c46dc9dadcb79c5e3981a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.701762 20336 tablet_service.cc:1467] Processing CreateTablet for tablet a84696c5572a45ca91c6e96514ea3d21 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:21.708328 20336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84696c5572a45ca91c6e96514ea3d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.709730 20259 tablet_service.cc:1467] Processing CreateTablet for tablet 74fbee6b4295483fa6ff7285ece93901 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:21.711014 20259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fbee6b4295483fa6ff7285ece93901. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.711942 20255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c439b8ab09aa4503a93dc9f98400a681. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.712920 20261 tablet_service.cc:1467] Processing CreateTablet for tablet a84696c5572a45ca91c6e96514ea3d21 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:21.714134 20261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84696c5572a45ca91c6e96514ea3d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.715466 20462 tablet_bootstrap.cc:492] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:21.715715 20262 tablet_service.cc:1467] Processing CreateTablet for tablet c9181f2ac504418faf8c6291311cd19b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:21.716929 20262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9181f2ac504418faf8c6291311cd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.722443 20462 tablet_bootstrap.cc:654] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:21.729792 20335 tablet_service.cc:1467] Processing CreateTablet for tablet 32907ca5566a44f9a3f284ba36e744ae (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:21.731051 20335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32907ca5566a44f9a3f284ba36e744ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.731750 20330 tablet_service.cc:1467] Processing CreateTablet for tablet c439b8ab09aa4503a93dc9f98400a681 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:21.733023 20330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c439b8ab09aa4503a93dc9f98400a681. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.736689 20331 tablet_service.cc:1467] Processing CreateTablet for tablet 43aad0e784aa4014b483577c8564bea4 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:21.737879 20331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43aad0e784aa4014b483577c8564bea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.741490 20333 tablet_service.cc:1467] Processing CreateTablet for tablet c59266f7526c46dc9dadcb79c5e3981a (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:21.742969 20334 tablet_service.cc:1467] Processing CreateTablet for tablet 74fbee6b4295483fa6ff7285ece93901 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:21.740098 20332 tablet_service.cc:1467] Processing CreateTablet for tablet 42b0e5bc9e0749c7b060f68f8855a77b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:21.745632 20333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59266f7526c46dc9dadcb79c5e3981a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.747167 20412 tablet_service.cc:1467] Processing CreateTablet for tablet 13960ac9c4094cf7b7a843d3a768433b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:21.748415 20412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13960ac9c4094cf7b7a843d3a768433b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.749022 20332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42b0e5bc9e0749c7b060f68f8855a77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.756872 20334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fbee6b4295483fa6ff7285ece93901. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.790509 20411 tablet_service.cc:1467] Processing CreateTablet for tablet c9181f2ac504418faf8c6291311cd19b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:21.792358 20411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9181f2ac504418faf8c6291311cd19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.794703 20410 tablet_service.cc:1467] Processing CreateTablet for tablet a84696c5572a45ca91c6e96514ea3d21 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:21.801667 20410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84696c5572a45ca91c6e96514ea3d21. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.797458 20409 tablet_service.cc:1467] Processing CreateTablet for tablet 32907ca5566a44f9a3f284ba36e744ae (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:21.805666 20409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32907ca5566a44f9a3f284ba36e744ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.810246 20408 tablet_service.cc:1467] Processing CreateTablet for tablet 74fbee6b4295483fa6ff7285ece93901 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:21.811900 20408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74fbee6b4295483fa6ff7285ece93901. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.815424 20407 tablet_service.cc:1467] Processing CreateTablet for tablet c59266f7526c46dc9dadcb79c5e3981a (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:21.817201 20407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c59266f7526c46dc9dadcb79c5e3981a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.855119 20406 tablet_service.cc:1467] Processing CreateTablet for tablet 43aad0e784aa4014b483577c8564bea4 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:21.856040 20412 tablet_service.cc:1467] Processing CreateTablet for tablet 42b0e5bc9e0749c7b060f68f8855a77b (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:21.858322 20405 tablet_service.cc:1467] Processing CreateTablet for tablet c439b8ab09aa4503a93dc9f98400a681 (DEFAULT_TABLE table=test-workload [id=e9e64b34efe9439dba98dbe888de7cd1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:21.859988 20406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43aad0e784aa4014b483577c8564bea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.864274 20464 tablet_bootstrap.cc:492] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:21.865665 20465 tablet_bootstrap.cc:492] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:21.869683 20405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c439b8ab09aa4503a93dc9f98400a681. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.875857 20462 tablet_bootstrap.cc:492] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:21.876382 20462 ts_tablet_manager.cc:1397] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.161s user 0.044s sys 0.066s
I20250112 20:59:21.879290 20465 tablet_bootstrap.cc:654] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:21.879122 20462 raft_consensus.cc:357] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:21.879810 20462 raft_consensus.cc:383] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:21.880115 20462 raft_consensus.cc:738] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:21.880884 20462 consensus_queue.cc:260] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:21.892992 20412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42b0e5bc9e0749c7b060f68f8855a77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:21.907649 20464 tablet_bootstrap.cc:654] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:21.946179 20464 tablet_bootstrap.cc:492] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:21.949959 20464 ts_tablet_manager.cc:1397] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.086s user 0.009s sys 0.005s
I20250112 20:59:21.953379 20464 raft_consensus.cc:357] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:21.955363 20298 heartbeater.cc:502] Master 127.15.23.190:45649 was elected leader, sending a full tablet report...
I20250112 20:59:21.955974 20462 ts_tablet_manager.cc:1428] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.079s user 0.030s sys 0.048s
I20250112 20:59:21.956315 20464 raft_consensus.cc:383] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:21.956780 20464 raft_consensus.cc:738] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:21.956916 20462 tablet_bootstrap.cc:492] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:21.957031 20465 tablet_bootstrap.cc:492] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:21.957629 20465 ts_tablet_manager.cc:1397] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.092s user 0.056s sys 0.027s
I20250112 20:59:21.957655 20464 consensus_queue.cc:260] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:21.960448 20465 raft_consensus.cc:357] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:21.961243 20465 raft_consensus.cc:383] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:21.961552 20465 raft_consensus.cc:738] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:21.962323 20465 consensus_queue.cc:260] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:21.964087 20462 tablet_bootstrap.cc:654] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:21.966562 20464 ts_tablet_manager.cc:1428] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.016s user 0.005s sys 0.007s
I20250112 20:59:21.967410 20373 heartbeater.cc:502] Master 127.15.23.190:45649 was elected leader, sending a full tablet report...
I20250112 20:59:21.967832 20464 tablet_bootstrap.cc:492] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:21.975198 20464 tablet_bootstrap.cc:654] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:21.979070 20465 ts_tablet_manager.cc:1428] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.021s user 0.008s sys 0.003s
I20250112 20:59:21.979863 20462 tablet_bootstrap.cc:492] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:21.979849 20447 heartbeater.cc:502] Master 127.15.23.190:45649 was elected leader, sending a full tablet report...
I20250112 20:59:21.980439 20465 tablet_bootstrap.cc:492] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:21.980647 20462 ts_tablet_manager.cc:1397] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.013s
I20250112 20:59:21.983343 20462 raft_consensus.cc:357] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:21.984088 20462 raft_consensus.cc:383] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:21.986726 20462 raft_consensus.cc:738] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:21.987433 20462 consensus_queue.cc:260] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:21.989651 20462 ts_tablet_manager.cc:1428] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.009s user 0.003s sys 0.003s
I20250112 20:59:21.990466 20462 tablet_bootstrap.cc:492] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:21.986375 20465 tablet_bootstrap.cc:654] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:21.998633 20462 tablet_bootstrap.cc:654] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.012704 20467 raft_consensus.cc:491] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.013367 20467 raft_consensus.cc:513] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.027648 20469 raft_consensus.cc:491] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.028321 20469 raft_consensus.cc:513] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.031209 20469 leader_election.cc:290] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.025270 20467 leader_election.cc:290] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.055474 20465 tablet_bootstrap.cc:492] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:22.056121 20465 ts_tablet_manager.cc:1397] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.076s user 0.031s sys 0.032s
I20250112 20:59:22.057546 20464 tablet_bootstrap.cc:492] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:22.059310 20464 ts_tablet_manager.cc:1397] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.092s user 0.039s sys 0.042s
I20250112 20:59:22.064118 20464 raft_consensus.cc:357] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.064544 20462 tablet_bootstrap.cc:492] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:22.065243 20462 ts_tablet_manager.cc:1397] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.075s user 0.038s sys 0.024s
I20250112 20:59:22.066965 20465 raft_consensus.cc:357] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.067754 20465 raft_consensus.cc:383] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.068044 20465 raft_consensus.cc:738] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.068835 20465 consensus_queue.cc:260] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.071141 20464 raft_consensus.cc:383] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.071451 20464 raft_consensus.cc:738] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.072167 20464 consensus_queue.cc:260] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.074728 20464 ts_tablet_manager.cc:1428] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20250112 20:59:22.075590 20464 tablet_bootstrap.cc:492] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:22.082921 20465 ts_tablet_manager.cc:1428] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.026s user 0.003s sys 0.003s
I20250112 20:59:22.083798 20465 tablet_bootstrap.cc:492] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:22.084437 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13960ac9c4094cf7b7a843d3a768433b" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:22.085237 20422 raft_consensus.cc:2463] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 0.
I20250112 20:59:22.086525 20236 leader_election.cc:304] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.087427 20467 raft_consensus.cc:2798] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:22.087762 20467 raft_consensus.cc:491] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.088109 20467 raft_consensus.cc:3054] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.096027 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13960ac9c4094cf7b7a843d3a768433b" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
W20250112 20:59:22.097183 20237 leader_election.cc:343] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:22.097851 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
W20250112 20:59:22.099766 20311 leader_election.cc:343] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:22.100070 20467 raft_consensus.cc:513] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.103030 20467 leader_election.cc:290] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.103211 20464 tablet_bootstrap.cc:654] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.104771 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13960ac9c4094cf7b7a843d3a768433b" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:22.105317 20422 raft_consensus.cc:3054] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.105454 20465 tablet_bootstrap.cc:654] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.108534 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13960ac9c4094cf7b7a843d3a768433b" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
W20250112 20:59:22.110652 20237 leader_election.cc:343] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:22.111212 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
W20250112 20:59:22.112702 20311 leader_election.cc:343] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:22.113248 20311 leader_election.cc:304] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142; no voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:22.113821 20462 raft_consensus.cc:357] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.114516 20462 raft_consensus.cc:383] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.114545 20469 raft_consensus.cc:2743] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:22.114800 20422 raft_consensus.cc:2463] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 1.
I20250112 20:59:22.114894 20462 raft_consensus.cc:738] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.116111 20236 leader_election.cc:304] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters: 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:22.116791 20467 raft_consensus.cc:2798] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.116676 20462 consensus_queue.cc:260] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.118450 20464 tablet_bootstrap.cc:492] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:22.119089 20464 ts_tablet_manager.cc:1397] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.044s user 0.017s sys 0.001s
I20250112 20:59:22.122357 20464 raft_consensus.cc:357] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.123667 20467 raft_consensus.cc:695] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:22.124508 20465 tablet_bootstrap.cc:492] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:22.124670 20464 raft_consensus.cc:383] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.125102 20465 ts_tablet_manager.cc:1397] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.041s user 0.005s sys 0.008s
I20250112 20:59:22.125142 20464 raft_consensus.cc:738] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.125259 20467 consensus_queue.cc:237] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.126026 20462 ts_tablet_manager.cc:1428] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.060s user 0.012s sys 0.001s
I20250112 20:59:22.126188 20464 consensus_queue.cc:260] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.127002 20462 tablet_bootstrap.cc:492] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:22.127671 20465 raft_consensus.cc:357] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.128402 20465 raft_consensus.cc:383] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.128832 20465 raft_consensus.cc:738] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.129657 20465 consensus_queue.cc:260] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.130189 20464 ts_tablet_manager.cc:1428] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20250112 20:59:22.131085 20464 tablet_bootstrap.cc:492] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:22.131789 20465 ts_tablet_manager.cc:1428] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250112 20:59:22.132675 20465 tablet_bootstrap.cc:492] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:22.136855 20464 tablet_bootstrap.cc:654] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.139498 20465 tablet_bootstrap.cc:654] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.147189 20167 catalog_manager.cc:5526] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:22.151502 20464 tablet_bootstrap.cc:492] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:22.151507 20462 tablet_bootstrap.cc:654] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.152184 20464 ts_tablet_manager.cc:1397] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.021s user 0.016s sys 0.000s
I20250112 20:59:22.155156 20464 raft_consensus.cc:357] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.156657 20464 raft_consensus.cc:383] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.157420 20464 raft_consensus.cc:738] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.158272 20464 consensus_queue.cc:260] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.159694 20465 tablet_bootstrap.cc:492] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:22.160094 20465 ts_tablet_manager.cc:1397] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.015s
I20250112 20:59:22.160521 20464 ts_tablet_manager.cc:1428] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250112 20:59:22.161563 20464 tablet_bootstrap.cc:492] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:22.162611 20465 raft_consensus.cc:357] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.163426 20465 raft_consensus.cc:383] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.163738 20465 raft_consensus.cc:738] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.164531 20465 consensus_queue.cc:260] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.166760 20465 ts_tablet_manager.cc:1428] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250112 20:59:22.167742 20465 tablet_bootstrap.cc:492] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:22.168924 20462 tablet_bootstrap.cc:492] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:22.169461 20462 ts_tablet_manager.cc:1397] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.008s
I20250112 20:59:22.169773 20464 tablet_bootstrap.cc:654] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.172130 20462 raft_consensus.cc:357] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.172933 20462 raft_consensus.cc:383] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.173858 20465 tablet_bootstrap.cc:654] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.173214 20462 raft_consensus.cc:738] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.174660 20462 consensus_queue.cc:260] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.179867 20462 ts_tablet_manager.cc:1428] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20250112 20:59:22.181102 20462 tablet_bootstrap.cc:492] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:22.190755 20462 tablet_bootstrap.cc:654] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.190930 20464 tablet_bootstrap.cc:492] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:22.191565 20464 ts_tablet_manager.cc:1397] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.015s
I20250112 20:59:22.194734 20464 raft_consensus.cc:357] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.195261 20465 tablet_bootstrap.cc:492] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:22.195403 20464 raft_consensus.cc:383] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.195719 20464 raft_consensus.cc:738] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.195796 20465 ts_tablet_manager.cc:1397] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.028s user 0.015s sys 0.011s
I20250112 20:59:22.196543 20462 tablet_bootstrap.cc:492] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:22.196388 20464 consensus_queue.cc:260] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.196955 20462 ts_tablet_manager.cc:1397] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.002s
I20250112 20:59:22.198827 20464 ts_tablet_manager.cc:1428] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20250112 20:59:22.198614 20465 raft_consensus.cc:357] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.199411 20465 raft_consensus.cc:383] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.199821 20464 tablet_bootstrap.cc:492] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:22.199718 20465 raft_consensus.cc:738] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.199708 20462 raft_consensus.cc:357] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.200644 20462 raft_consensus.cc:383] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.200989 20462 raft_consensus.cc:738] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.200861 20465 consensus_queue.cc:260] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.201730 20462 consensus_queue.cc:260] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.203214 20465 ts_tablet_manager.cc:1428] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:59:22.204154 20462 ts_tablet_manager.cc:1428] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:59:22.204197 20465 tablet_bootstrap.cc:492] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:22.205333 20462 tablet_bootstrap.cc:492] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:22.206252 20464 tablet_bootstrap.cc:654] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.211434 20462 tablet_bootstrap.cc:654] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.211795 20465 tablet_bootstrap.cc:654] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.221681 20469 raft_consensus.cc:491] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.222361 20462 tablet_bootstrap.cc:492] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:22.222131 20469 raft_consensus.cc:513] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.222841 20462 ts_tablet_manager.cc:1397] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.014s
I20250112 20:59:22.224012 20469 leader_election.cc:290] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.225354 20462 raft_consensus.cc:357] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.225821 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:22.227819 20462 raft_consensus.cc:383] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.228204 20462 raft_consensus.cc:738] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.225979 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
W20250112 20:59:22.228314 20311 leader_election.cc:343] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:22.229072 20462 consensus_queue.cc:260] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
W20250112 20:59:22.229593 20311 leader_election.cc:343] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:22.230016 20311 leader_election.cc:304] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142; no voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:22.230473 20465 tablet_bootstrap.cc:492] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:22.231065 20469 raft_consensus.cc:2743] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:22.231093 20465 ts_tablet_manager.cc:1397] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.003s
I20250112 20:59:22.233150 20462 ts_tablet_manager.cc:1428] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20250112 20:59:22.234154 20462 tablet_bootstrap.cc:492] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:22.235495 20465 raft_consensus.cc:357] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.236608 20465 raft_consensus.cc:383] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.237046 20465 raft_consensus.cc:738] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.238112 20465 consensus_queue.cc:260] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.241436 20465 ts_tablet_manager.cc:1428] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20250112 20:59:22.241905 20462 tablet_bootstrap.cc:654] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.242659 20465 tablet_bootstrap.cc:492] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
W20250112 20:59:22.245931 20219 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:59:22.251289 20465 tablet_bootstrap.cc:654] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.255425 20462 tablet_bootstrap.cc:492] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:22.255937 20462 ts_tablet_manager.cc:1397] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.005s
I20250112 20:59:22.258754 20462 raft_consensus.cc:357] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.259611 20462 raft_consensus.cc:383] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.259938 20462 raft_consensus.cc:738] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.260747 20462 consensus_queue.cc:260] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.265027 20462 ts_tablet_manager.cc:1428] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
I20250112 20:59:22.266112 20462 tablet_bootstrap.cc:492] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:22.267352 20465 tablet_bootstrap.cc:492] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:22.268113 20465 ts_tablet_manager.cc:1397] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.026s user 0.018s sys 0.002s
I20250112 20:59:22.271714 20465 raft_consensus.cc:357] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.272729 20465 raft_consensus.cc:383] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.272367 20462 tablet_bootstrap.cc:654] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.273167 20465 raft_consensus.cc:738] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.274400 20465 consensus_queue.cc:260] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.278156 20465 ts_tablet_manager.cc:1428] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250112 20:59:22.278630 20464 tablet_bootstrap.cc:492] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:22.278730 20462 tablet_bootstrap.cc:492] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:22.279150 20465 tablet_bootstrap.cc:492] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:22.279181 20464 ts_tablet_manager.cc:1397] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.080s user 0.025s sys 0.052s
I20250112 20:59:22.279192 20462 ts_tablet_manager.cc:1397] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20250112 20:59:22.281957 20462 raft_consensus.cc:357] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.282413 20462 raft_consensus.cc:383] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.282614 20462 raft_consensus.cc:738] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.283083 20462 consensus_queue.cc:260] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.284817 20462 ts_tablet_manager.cc:1428] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20250112 20:59:22.285128 20465 tablet_bootstrap.cc:654] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.285576 20462 tablet_bootstrap.cc:492] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:22.290709 20462 tablet_bootstrap.cc:654] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.293798 20464 raft_consensus.cc:357] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.294577 20464 raft_consensus.cc:383] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.294860 20464 raft_consensus.cc:738] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.295666 20462 tablet_bootstrap.cc:492] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:22.296053 20462 ts_tablet_manager.cc:1397] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20250112 20:59:22.295524 20464 consensus_queue.cc:260] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.297801 20464 ts_tablet_manager.cc:1428] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.018s user 0.005s sys 0.001s
I20250112 20:59:22.297866 20465 tablet_bootstrap.cc:492] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:22.298292 20465 ts_tablet_manager.cc:1397] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20250112 20:59:22.298790 20464 tablet_bootstrap.cc:492] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:22.300447 20465 raft_consensus.cc:357] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.301218 20465 raft_consensus.cc:383] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.301512 20465 raft_consensus.cc:738] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.302243 20465 consensus_queue.cc:260] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.302810 20462 raft_consensus.cc:357] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.303495 20462 raft_consensus.cc:383] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.303755 20462 raft_consensus.cc:738] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.304143 20464 tablet_bootstrap.cc:654] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.304427 20462 consensus_queue.cc:260] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.306569 20465 ts_tablet_manager.cc:1428] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20250112 20:59:22.306838 20462 ts_tablet_manager.cc:1428] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:59:22.307518 20465 tablet_bootstrap.cc:492] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:22.310843 20464 tablet_bootstrap.cc:492] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:22.311343 20464 ts_tablet_manager.cc:1397] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.003s
I20250112 20:59:22.312814 20465 tablet_bootstrap.cc:654] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.313798 20464 raft_consensus.cc:357] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.314572 20464 raft_consensus.cc:383] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.314858 20464 raft_consensus.cc:738] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.315467 20464 consensus_queue.cc:260] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.317579 20470 raft_consensus.cc:491] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.318053 20470 raft_consensus.cc:513] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.319603 20464 ts_tablet_manager.cc:1428] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.008s user 0.003s sys 0.002s
I20250112 20:59:22.320302 20470 leader_election.cc:290] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.320495 20464 tablet_bootstrap.cc:492] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:22.326491 20464 tablet_bootstrap.cc:654] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.335795 20470 raft_consensus.cc:491] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.336365 20470 raft_consensus.cc:513] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.338454 20470 leader_election.cc:290] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.341251 20464 tablet_bootstrap.cc:492] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:22.341703 20464 ts_tablet_manager.cc:1397] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.012s
I20250112 20:59:22.341276 20465 tablet_bootstrap.cc:492] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:22.342365 20465 ts_tablet_manager.cc:1397] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.008s
I20250112 20:59:22.343951 20464 raft_consensus.cc:357] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.344594 20464 raft_consensus.cc:383] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.344828 20465 raft_consensus.cc:357] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.345590 20465 raft_consensus.cc:383] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.345890 20465 raft_consensus.cc:738] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.344884 20464 raft_consensus.cc:738] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.346617 20465 consensus_queue.cc:260] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.346841 20464 consensus_queue.cc:260] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.347803 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.348783 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:22.349090 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.348444 20273 raft_consensus.cc:2463] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.349845 20272 raft_consensus.cc:2463] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.349380 20464 ts_tablet_manager.cc:1428] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:59:22.350374 20347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:22.351150 20464 tablet_bootstrap.cc:492] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:22.351009 20385 leader_election.cc:304] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.351796 20385 leader_election.cc:304] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.352878 20470 raft_consensus.cc:2798] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:22.353199 20470 raft_consensus.cc:491] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.353490 20470 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.349470 20348 raft_consensus.cc:2463] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.354420 20465 ts_tablet_manager.cc:1428] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
W20250112 20:59:22.355273 20386 leader_election.cc:343] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:22.356575 20464 tablet_bootstrap.cc:654] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:22.360576 20504 raft_consensus.cc:2798] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:22.361030 20504 raft_consensus.cc:491] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.361346 20504 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.361871 20470 raft_consensus.cc:513] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.365097 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.365849 20272 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.369012 20504 raft_consensus.cc:513] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.371564 20504 leader_election.cc:290] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.372025 20504 raft_consensus.cc:491] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.372506 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.372412 20504 raft_consensus.cc:513] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.373348 20273 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.374709 20504 leader_election.cc:290] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.375732 20347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32907ca5566a44f9a3f284ba36e744ae" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:22.376384 20347 raft_consensus.cc:2463] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.372515 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:22.379158 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32907ca5566a44f9a3f284ba36e744ae" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.379747 20386 leader_election.cc:304] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:22.380115 20271 raft_consensus.cc:2463] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.380620 20506 raft_consensus.cc:2798] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20250112 20:59:22.380643 20386 leader_election.cc:343] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:22.381181 20506 raft_consensus.cc:491] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.381560 20506 raft_consensus.cc:3054] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.383991 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:22.384563 20348 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.385828 20272 raft_consensus.cc:2463] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.386786 20385 leader_election.cc:304] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.387598 20504 raft_consensus.cc:2798] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.387984 20273 raft_consensus.cc:2463] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.389619 20385 leader_election.cc:304] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters: 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:22.389016 20504 raft_consensus.cc:695] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.391149 20504 consensus_queue.cc:237] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.391491 20469 raft_consensus.cc:491] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.391362 20464 tablet_bootstrap.cc:492] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:22.392280 20464 ts_tablet_manager.cc:1397] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.041s user 0.018s sys 0.015s
I20250112 20:59:22.392046 20469 raft_consensus.cc:513] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.393929 20469 leader_election.cc:290] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.395304 20464 raft_consensus.cc:357] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.395740 20348 raft_consensus.cc:2463] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.395982 20464 raft_consensus.cc:383] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:22.396358 20507 raft_consensus.cc:2798] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.388909 20470 leader_election.cc:290] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.400987 20507 raft_consensus.cc:695] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.401054 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13960ac9c4094cf7b7a843d3a768433b" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.397485 20464 raft_consensus.cc:738] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:22.401793 20507 consensus_queue.cc:237] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.402287 20464 consensus_queue.cc:260] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.401964 20506 raft_consensus.cc:513] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.404945 20464 ts_tablet_manager.cc:1428] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.012s user 0.008s sys 0.001s
I20250112 20:59:22.406026 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13960ac9c4094cf7b7a843d3a768433b" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:22.406746 20422 raft_consensus.cc:2388] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a2063dd1fee4cb48fe32bf917055142 in current term 1: Already voted for candidate 792c9d15fc44406b9058909c5a860f96 in this term.
I20250112 20:59:22.407684 20311 leader_election.cc:304] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142; no voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:22.408849 20469 raft_consensus.cc:3054] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.409885 20506 leader_election.cc:290] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.410223 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32907ca5566a44f9a3f284ba36e744ae" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.410472 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32907ca5566a44f9a3f284ba36e744ae" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:22.401150 20510 raft_consensus.cc:491] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.411414 20348 raft_consensus.cc:3054] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.411588 20510 raft_consensus.cc:513] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.415812 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fbee6b4295483fa6ff7285ece93901" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.415946 20469 raft_consensus.cc:2743] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:22.416540 20272 raft_consensus.cc:2463] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.418650 20347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fbee6b4295483fa6ff7285ece93901" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:22.418758 20348 raft_consensus.cc:2463] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.419245 20347 raft_consensus.cc:2463] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.419723 20386 leader_election.cc:304] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:22.420199 20385 leader_election.cc:304] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.410929 20273 raft_consensus.cc:3054] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.420832 20507 raft_consensus.cc:2798] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.420993 20506 raft_consensus.cc:2798] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:22.421504 20506 raft_consensus.cc:491] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.421584 20507 raft_consensus.cc:695] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.421927 20506 raft_consensus.cc:3054] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.424551 20507 consensus_queue.cc:237] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.425727 20273 raft_consensus.cc:2463] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.427165 20469 raft_consensus.cc:491] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.420364 20510 leader_election.cc:290] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.427659 20469 raft_consensus.cc:513] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.429580 20469 leader_election.cc:290] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.429843 20167 catalog_manager.cc:5526] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:22.430927 20506 raft_consensus.cc:513] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.431517 20510 raft_consensus.cc:491] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.431977 20510 raft_consensus.cc:513] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.432852 20506 leader_election.cc:290] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.433470 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b0e5bc9e0749c7b060f68f8855a77b" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:22.433682 20510 leader_election.cc:290] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.433830 20347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fbee6b4295483fa6ff7285ece93901" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:22.434123 20422 raft_consensus.cc:2463] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 0.
I20250112 20:59:22.431535 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b0e5bc9e0749c7b060f68f8855a77b" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.434423 20347 raft_consensus.cc:3054] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.434907 20273 raft_consensus.cc:2463] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 0.
I20250112 20:59:22.435022 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:22.435640 20348 raft_consensus.cc:2463] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.435951 20311 leader_election.cc:304] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.436530 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fbee6b4295483fa6ff7285ece93901" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.436852 20469 raft_consensus.cc:2798] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:22.437110 20272 raft_consensus.cc:3054] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.437278 20469 raft_consensus.cc:491] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.437731 20469 raft_consensus.cc:3054] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.439970 20386 leader_election.cc:304] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:22.441488 20347 raft_consensus.cc:2463] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.442415 20386 leader_election.cc:304] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:22.443130 20510 raft_consensus.cc:2798] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:22.443498 20510 raft_consensus.cc:491] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.443856 20510 raft_consensus.cc:3054] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.444880 20507 raft_consensus.cc:2798] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.445767 20507 raft_consensus.cc:695] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.446483 20507 consensus_queue.cc:237] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.447800 20272 raft_consensus.cc:2463] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.449306 20167 catalog_manager.cc:5526] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:22.450971 20167 catalog_manager.cc:5526] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:22.453225 20510 raft_consensus.cc:513] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.454057 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.449800 20469 raft_consensus.cc:513] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.454775 20273 raft_consensus.cc:2463] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.455847 20510 leader_election.cc:290] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.456115 20469 leader_election.cc:290] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.456708 20469 raft_consensus.cc:491] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.456876 20347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:22.457177 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.457908 20272 raft_consensus.cc:3054] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.457463 20347 raft_consensus.cc:3054] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.457774 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b0e5bc9e0749c7b060f68f8855a77b" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:22.458529 20422 raft_consensus.cc:3054] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.457108 20469 raft_consensus.cc:513] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.464251 20272 raft_consensus.cc:2463] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.464334 20347 raft_consensus.cc:2463] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.464576 20422 raft_consensus.cc:2463] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 1.
I20250112 20:59:22.465291 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42b0e5bc9e0749c7b060f68f8855a77b" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.465452 20386 leader_election.cc:304] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:22.465931 20273 raft_consensus.cc:3054] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.465986 20469 leader_election.cc:290] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.466289 20510 raft_consensus.cc:2798] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.466748 20510 raft_consensus.cc:695] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.467499 20311 leader_election.cc:304] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:22.468421 20469 raft_consensus.cc:2798] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.470340 20469 raft_consensus.cc:695] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:22.470572 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c439b8ab09aa4503a93dc9f98400a681" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.467541 20510 consensus_queue.cc:237] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.471414 20272 raft_consensus.cc:2463] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 0.
I20250112 20:59:22.469053 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c439b8ab09aa4503a93dc9f98400a681" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:22.472184 20422 raft_consensus.cc:2463] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 0.
I20250112 20:59:22.472319 20273 raft_consensus.cc:2463] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 1.
I20250112 20:59:22.472721 20311 leader_election.cc:304] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.472757 20469 consensus_queue.cc:237] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.477780 20515 raft_consensus.cc:2798] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:22.478248 20515 raft_consensus.cc:491] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.478668 20515 raft_consensus.cc:3054] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.486446 20515 raft_consensus.cc:513] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.488862 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c439b8ab09aa4503a93dc9f98400a681" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.489442 20272 raft_consensus.cc:3054] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.488664 20167 catalog_manager.cc:5526] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:22.488955 20515 leader_election.cc:290] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.490595 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c439b8ab09aa4503a93dc9f98400a681" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:22.491436 20422 raft_consensus.cc:3054] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.490676 20168 catalog_manager.cc:5526] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:22.492715 20168 catalog_manager.cc:5526] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:22.495846 20272 raft_consensus.cc:2463] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 1.
I20250112 20:59:22.496857 20311 leader_election.cc:304] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.498381 20422 raft_consensus.cc:2463] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 1.
I20250112 20:59:22.499190 20515 raft_consensus.cc:2798] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.499665 20515 raft_consensus.cc:695] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:22.500454 20515 consensus_queue.cc:237] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.508090 20167 catalog_manager.cc:5526] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:22.514701 20467 consensus_queue.cc:1035] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.525864 20482 consensus_queue.cc:1035] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:59:22.627411 20510 raft_consensus.cc:491] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:22.627840 20510 raft_consensus.cc:513] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.629290 20510 leader_election.cc:290] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.630141 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:22.630211 20347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:22.630764 20272 raft_consensus.cc:2463] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.630864 20347 raft_consensus.cc:2463] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:22.631558 20385 leader_election.cc:304] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.632220 20510 raft_consensus.cc:2798] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:22.632553 20510 raft_consensus.cc:491] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:22.632828 20510 raft_consensus.cc:3054] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.637317 20510 raft_consensus.cc:513] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.638679 20510 leader_election.cc:290] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:22.639405 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.639643 20347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:22.639904 20272 raft_consensus.cc:3054] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.640151 20347 raft_consensus.cc:3054] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.644860 20272 raft_consensus.cc:2463] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.644886 20347 raft_consensus.cc:2463] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:22.645743 20385 leader_election.cc:304] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.646322 20510 raft_consensus.cc:2798] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:22.646744 20510 raft_consensus.cc:695] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.647446 20510 consensus_queue.cc:237] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.653558 20168 catalog_manager.cc:5526] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:22.764904 20422 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
mode: GRACEFUL
new_leader_uuid: "792c9d15fc44406b9058909c5a860f96"
from {username='slave'} at 127.0.0.1:46570
I20250112 20:59:22.765441 20422 raft_consensus.cc:604] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Received request to transfer leadership to TS 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:22.769572 20422 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
mode: GRACEFUL
new_leader_uuid: "792c9d15fc44406b9058909c5a860f96"
from {username='slave'} at 127.0.0.1:46570
I20250112 20:59:22.770036 20422 raft_consensus.cc:604] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Received request to transfer leadership to TS 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:22.774039 20422 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "32907ca5566a44f9a3f284ba36e744ae"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
mode: GRACEFUL
new_leader_uuid: "792c9d15fc44406b9058909c5a860f96"
from {username='slave'} at 127.0.0.1:46570
I20250112 20:59:22.774457 20422 raft_consensus.cc:604] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Received request to transfer leadership to TS 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:22.778476 20422 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "74fbee6b4295483fa6ff7285ece93901"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
mode: GRACEFUL
new_leader_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:46570
I20250112 20:59:22.778908 20422 raft_consensus.cc:604] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Received request to transfer leadership to TS 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:22.782850 20422 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
mode: GRACEFUL
new_leader_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:46570
I20250112 20:59:22.783253 20422 raft_consensus.cc:604] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Received request to transfer leadership to TS 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:22.789492 20422 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
mode: GRACEFUL
new_leader_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:46570
I20250112 20:59:22.789952 20422 raft_consensus.cc:604] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Received request to transfer leadership to TS 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:22.792691 20510 consensus_queue.cc:1035] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.801694 20347 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:22.804291 20348 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c439b8ab09aa4503a93dc9f98400a681"
dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
mode: GRACEFUL
new_leader_uuid: "0c62e542bb824359a6a7a83070a62b61"
from {username='slave'} at 127.0.0.1:54160
I20250112 20:59:22.804939 20348 raft_consensus.cc:604] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Received request to transfer leadership to TS 0c62e542bb824359a6a7a83070a62b61
I20250112 20:59:22.810845 20510 consensus_queue.cc:1035] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.816900 20506 raft_consensus.cc:988] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61: : Instructing follower 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:22.817255 20510 raft_consensus.cc:1076] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Signalling peer 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:22.818536 20347 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4"
dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:54156
I20250112 20:59:22.819105 20347 raft_consensus.cc:491] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:22.819463 20347 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.824504 20347 raft_consensus.cc:513] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.826174 20347 leader_election.cc:290] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.825933 20510 consensus_queue.cc:1035] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.826717 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.827121 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:22.827339 20272 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.827734 20422 raft_consensus.cc:3049] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:22.828047 20422 raft_consensus.cc:738] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.828835 20422 consensus_queue.cc:260] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.829942 20422 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.832851 20506 raft_consensus.cc:988] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61: : Instructing follower 792c9d15fc44406b9058909c5a860f96 to start an election
I20250112 20:59:22.833247 20506 raft_consensus.cc:1076] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Signalling peer 792c9d15fc44406b9058909c5a860f96 to start an election
I20250112 20:59:22.833658 20272 raft_consensus.cc:2463] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:22.835408 20311 leader_election.cc:304] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.836082 20273 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21"
dest_uuid: "792c9d15fc44406b9058909c5a860f96"
from {username='slave'} at 127.0.0.1:45108
I20250112 20:59:22.836253 20515 raft_consensus.cc:2798] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:22.836637 20273 raft_consensus.cc:491] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:22.836781 20515 raft_consensus.cc:695] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:22.837007 20273 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.837553 20515 consensus_queue.cc:237] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.843130 20273 raft_consensus.cc:513] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.844446 20272 raft_consensus.cc:1235] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Rejecting Update request from peer 0c62e542bb824359a6a7a83070a62b61 for earlier term 1. Current term is 2. Ops: []
I20250112 20:59:22.844750 20504 consensus_queue.cc:1035] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.845331 20273 leader_election.cc:290] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.845789 20470 consensus_queue.cc:1046] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.846302 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:22.846742 20504 raft_consensus.cc:3049] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:22.846585 20470 consensus_queue.cc:1035] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.847138 20504 raft_consensus.cc:738] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.847663 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:22.848289 20348 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.847934 20504 consensus_queue.cc:260] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.849146 20504 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.851680 20422 raft_consensus.cc:2463] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:22.855474 20348 raft_consensus.cc:2463] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 2.
I20250112 20:59:22.857578 20347 raft_consensus.cc:1235] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Rejecting Update request from peer 0c62e542bb824359a6a7a83070a62b61 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20250112 20:59:22.857412 20237 leader_election.cc:304] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.858403 20467 raft_consensus.cc:2798] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:22.858893 20467 raft_consensus.cc:695] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:22.861114 20510 consensus_queue.cc:1035] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.861431 20421 raft_consensus.cc:2463] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 2.
I20250112 20:59:22.867885 20504 raft_consensus.cc:988] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61: : Instructing follower 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:22.868894 20470 raft_consensus.cc:1076] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Signalling peer 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:22.870308 20347 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a"
dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:54156
I20250112 20:59:22.870723 20347 raft_consensus.cc:491] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:22.870985 20347 raft_consensus.cc:3054] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.873556 20467 consensus_queue.cc:237] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.875530 20504 consensus_queue.cc:1035] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.878860 20347 raft_consensus.cc:513] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.879210 20470 consensus_queue.cc:1035] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.882493 20348 raft_consensus.cc:1235] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Rejecting Update request from peer 0c62e542bb824359a6a7a83070a62b61 for earlier term 1. Current term is 2. Ops: []
I20250112 20:59:22.883013 20347 leader_election.cc:290] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.884616 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.885190 20271 raft_consensus.cc:3054] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.885048 20169 catalog_manager.cc:5526] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 1 to 2, leader changed from 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131) to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:22.888463 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:22.888270 20167 catalog_manager.cc:5526] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 1 to 2, leader changed from 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131) to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:22.889212 20421 raft_consensus.cc:3049] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:22.889652 20421 raft_consensus.cc:738] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.890647 20421 consensus_queue.cc:260] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.891839 20421 raft_consensus.cc:3054] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.890661 20507 consensus_queue.cc:1035] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250112 20:59:22.899441 20421 raft_consensus.cc:2463] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:22.900530 20311 leader_election.cc:304] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:22.901005 20271 raft_consensus.cc:2463] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:22.901335 20469 raft_consensus.cc:2798] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:22.901852 20469 raft_consensus.cc:695] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:22.902736 20469 consensus_queue.cc:237] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.901669 20515 consensus_queue.cc:1035] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.901727 20273 raft_consensus.cc:1235] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Rejecting Update request from peer 0c62e542bb824359a6a7a83070a62b61 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20250112 20:59:22.905722 20470 raft_consensus.cc:988] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61: : Instructing follower 792c9d15fc44406b9058909c5a860f96 to start an election
I20250112 20:59:22.906205 20470 raft_consensus.cc:1076] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Signalling peer 792c9d15fc44406b9058909c5a860f96 to start an election
I20250112 20:59:22.909241 20272 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "32907ca5566a44f9a3f284ba36e744ae"
dest_uuid: "792c9d15fc44406b9058909c5a860f96"
from {username='slave'} at 127.0.0.1:45108
I20250112 20:59:22.909770 20272 raft_consensus.cc:491] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:22.910110 20272 raft_consensus.cc:3054] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.915583 20272 raft_consensus.cc:513] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.919726 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32907ca5566a44f9a3f284ba36e744ae" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:22.920295 20348 raft_consensus.cc:3054] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.922720 20272 leader_election.cc:290] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.924726 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32907ca5566a44f9a3f284ba36e744ae" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:22.925597 20421 raft_consensus.cc:3049] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:22.924647 20168 catalog_manager.cc:5526] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 1 to 2, leader changed from 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131) to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:22.925890 20421 raft_consensus.cc:738] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.926028 20470 raft_consensus.cc:988] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61: : Instructing follower 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:22.926419 20470 raft_consensus.cc:1076] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Signalling peer 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:22.926645 20421 consensus_queue.cc:260] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.929353 20345 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "74fbee6b4295483fa6ff7285ece93901"
dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:54156
I20250112 20:59:22.930987 20345 raft_consensus.cc:491] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:22.931332 20345 raft_consensus.cc:3054] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.938604 20470 consensus_queue.cc:1035] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:59:22.938715 20345 raft_consensus.cc:513] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.928011 20421 raft_consensus.cc:3054] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.945433 20422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fbee6b4295483fa6ff7285ece93901" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:22.946225 20422 raft_consensus.cc:3049] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:22.946573 20422 raft_consensus.cc:738] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:22.948971 20273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fbee6b4295483fa6ff7285ece93901" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:22.950747 20422 consensus_queue.cc:260] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.952096 20422 raft_consensus.cc:3054] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:22.953114 20514 consensus_queue.cc:1035] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.953553 20345 leader_election.cc:290] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:22.958506 20421 raft_consensus.cc:2463] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 2.
I20250112 20:59:22.959481 20348 raft_consensus.cc:2463] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 2.
I20250112 20:59:22.964340 20236 leader_election.cc:304] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:22.965420 20514 consensus_queue.cc:1035] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.965917 20482 raft_consensus.cc:2798] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:22.966166 20515 consensus_queue.cc:1035] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:22.966449 20482 raft_consensus.cc:695] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:22.967232 20482 consensus_queue.cc:237] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.972201 20422 raft_consensus.cc:2463] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:22.973160 20311 leader_election.cc:304] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters: 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:22.974050 20514 raft_consensus.cc:2798] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:22.974484 20514 raft_consensus.cc:695] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:22.975267 20514 consensus_queue.cc:237] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:22.980674 20167 catalog_manager.cc:5526] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 1 to 2, leader changed from 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131) to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:22.992983 20515 raft_consensus.cc:988] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142: : Instructing follower 0c62e542bb824359a6a7a83070a62b61 to start an election
I20250112 20:59:22.993486 20515 raft_consensus.cc:1076] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Signalling peer 0c62e542bb824359a6a7a83070a62b61 to start an election
I20250112 20:59:22.998150 20420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c439b8ab09aa4503a93dc9f98400a681"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
from {username='slave'} at 127.0.0.1:46558
I20250112 20:59:22.997751 20168 catalog_manager.cc:5526] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 1 to 2, leader changed from 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131) to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:22.998704 20420 raft_consensus.cc:491] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:22.999076 20420 raft_consensus.cc:3054] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:23.005822 20420 raft_consensus.cc:513] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:23.008144 20420 leader_election.cc:290] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 2 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:23.008479 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c439b8ab09aa4503a93dc9f98400a681" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:23.008801 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c439b8ab09aa4503a93dc9f98400a681" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:23.009227 20271 raft_consensus.cc:3054] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:23.009745 20345 raft_consensus.cc:3049] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:23.010087 20345 raft_consensus.cc:738] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:23.010828 20345 consensus_queue.cc:260] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:23.011683 20345 raft_consensus.cc:3054] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:23.014109 20271 raft_consensus.cc:2463] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 2.
I20250112 20:59:23.014891 20385 leader_election.cc:304] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:23.015481 20470 raft_consensus.cc:2798] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:23.015952 20470 raft_consensus.cc:695] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 2 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:23.016189 20345 raft_consensus.cc:2463] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 2.
I20250112 20:59:23.016816 20470 consensus_queue.cc:237] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:23.022998 20167 catalog_manager.cc:5526] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 1 to 2, leader changed from 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130) to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 2 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:23.043008 20470 consensus_queue.cc:1035] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:23.052309 20470 consensus_queue.cc:1035] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:23.059576 20504 raft_consensus.cc:988] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61: : Instructing follower 792c9d15fc44406b9058909c5a860f96 to start an election
I20250112 20:59:23.060001 20470 raft_consensus.cc:1076] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Signalling peer 792c9d15fc44406b9058909c5a860f96 to start an election
I20250112 20:59:23.061254 20271 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b"
dest_uuid: "792c9d15fc44406b9058909c5a860f96"
from {username='slave'} at 127.0.0.1:45108
I20250112 20:59:23.061702 20271 raft_consensus.cc:491] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:23.061959 20271 raft_consensus.cc:3054] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:23.066373 20271 raft_consensus.cc:513] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:23.067795 20271 leader_election.cc:290] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:23.068938 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:23.069195 20420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9181f2ac504418faf8c6291311cd19b" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:23.069602 20345 raft_consensus.cc:3054] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:23.069765 20420 raft_consensus.cc:3049] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:23.070118 20420 raft_consensus.cc:738] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:23.070886 20420 consensus_queue.cc:260] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:23.072073 20420 raft_consensus.cc:3054] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:23.075857 20345 raft_consensus.cc:2463] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 2.
I20250112 20:59:23.076885 20237 leader_election.cc:304] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:23.077503 20482 raft_consensus.cc:2798] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:23.077934 20420 raft_consensus.cc:2463] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 2.
I20250112 20:59:23.078038 20482 raft_consensus.cc:695] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:23.078990 20482 consensus_queue.cc:237] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:23.084411 20168 catalog_manager.cc:5526] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 1 to 2, leader changed from 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131) to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:23.341892 20271 raft_consensus.cc:1270] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.343034 20514 consensus_queue.cc:1035] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.350845 20420 raft_consensus.cc:1270] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.352401 20515 consensus_queue.cc:1035] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.360649 20271 raft_consensus.cc:3054] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:23.361563 20345 raft_consensus.cc:1270] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Refusing update from remote peer 792c9d15fc44406b9058909c5a860f96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.363510 20483 consensus_queue.cc:1035] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:23.366204 20420 raft_consensus.cc:1270] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 792c9d15fc44406b9058909c5a860f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.367456 20483 consensus_queue.cc:1035] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.368062 20271 raft_consensus.cc:1270] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.369728 20515 consensus_queue.cc:1035] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.385586 20420 raft_consensus.cc:1270] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.387110 20533 consensus_queue.cc:1035] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:59:23.435935 20271 raft_consensus.cc:1270] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Refusing update from remote peer 0c62e542bb824359a6a7a83070a62b61: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.436869 20470 consensus_queue.cc:1035] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.440351 20345 raft_consensus.cc:1270] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Refusing update from remote peer 0c62e542bb824359a6a7a83070a62b61: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.441540 20504 consensus_queue.cc:1035] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.468385 20345 raft_consensus.cc:1270] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Refusing update from remote peer 792c9d15fc44406b9058909c5a860f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.469405 20483 consensus_queue.cc:1035] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.475461 20420 raft_consensus.cc:1270] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 792c9d15fc44406b9058909c5a860f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.477123 20483 consensus_queue.cc:1035] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.494235 20420 raft_consensus.cc:1270] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 792c9d15fc44406b9058909c5a860f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.495029 20271 raft_consensus.cc:1270] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.495352 20483 consensus_queue.cc:1035] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.496172 20515 consensus_queue.cc:1035] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:23.498426 20421 raft_consensus.cc:1270] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.499676 20515 consensus_queue.cc:1035] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:23.505409 20345 raft_consensus.cc:1270] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Refusing update from remote peer 792c9d15fc44406b9058909c5a860f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:23.506728 20483 consensus_queue.cc:1035] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:25.806753 15454 test_util.cc:274] Using random seed: -1704971275
I20250112 20:59:25.835911 20167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37374:
name: "soft_deleted_table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250112 20:59:25.838586 20167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250112 20:59:25.888465 20257 tablet_service.cc:1467] Processing CreateTablet for tablet cc6fc6d0d5b5465da3c14a81f03e98e4 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:25.888871 20331 tablet_service.cc:1467] Processing CreateTablet for tablet cc6fc6d0d5b5465da3c14a81f03e98e4 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:25.890233 20331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc6fc6d0d5b5465da3c14a81f03e98e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.890291 20257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc6fc6d0d5b5465da3c14a81f03e98e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.891731 20338 tablet_service.cc:1467] Processing CreateTablet for tablet be0a0ef67bf6496e94624895d52de105 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:25.893047 20338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0a0ef67bf6496e94624895d52de105. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.898252 20335 tablet_service.cc:1467] Processing CreateTablet for tablet fe780069d490455e9dc46384034562c3 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:25.899631 20335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe780069d490455e9dc46384034562c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.899950 20405 tablet_service.cc:1467] Processing CreateTablet for tablet cc6fc6d0d5b5465da3c14a81f03e98e4 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION VALUES < 238609294
I20250112 20:59:25.901242 20405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc6fc6d0d5b5465da3c14a81f03e98e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.905735 20406 tablet_service.cc:1467] Processing CreateTablet for tablet be0a0ef67bf6496e94624895d52de105 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:25.906987 20406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0a0ef67bf6496e94624895d52de105. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.906858 20330 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcf8035cd0243398d1e294066f0e9b0 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:25.907127 20332 tablet_service.cc:1467] Processing CreateTablet for tablet 2f602ef67f28479caf359326135a508d (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:25.908128 20330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bcf8035cd0243398d1e294066f0e9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.909758 20333 tablet_service.cc:1467] Processing CreateTablet for tablet 910996064780419b92c4e87697377d1a (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:25.910923 20407 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcf8035cd0243398d1e294066f0e9b0 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:25.912142 20407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bcf8035cd0243398d1e294066f0e9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.913669 20411 tablet_service.cc:1467] Processing CreateTablet for tablet fe780069d490455e9dc46384034562c3 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:25.914848 20411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe780069d490455e9dc46384034562c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.917807 20337 tablet_service.cc:1467] Processing CreateTablet for tablet 44c4b3bbc0854317a9a279a899bf01a7 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:25.920513 20255 tablet_service.cc:1467] Processing CreateTablet for tablet be0a0ef67bf6496e94624895d52de105 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20250112 20:59:25.921228 20260 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcf8035cd0243398d1e294066f0e9b0 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20250112 20:59:25.910924 20334 tablet_service.cc:1467] Processing CreateTablet for tablet e81ee341c3274675b9ce0eb4f2093272 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:25.923132 20332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f602ef67f28479caf359326135a508d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.923038 20258 tablet_service.cc:1467] Processing CreateTablet for tablet fe780069d490455e9dc46384034562c3 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20250112 20:59:25.923331 20262 tablet_service.cc:1467] Processing CreateTablet for tablet 44c4b3bbc0854317a9a279a899bf01a7 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:25.921722 20255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be0a0ef67bf6496e94624895d52de105. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.924876 20337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c4b3bbc0854317a9a279a899bf01a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.925846 20260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bcf8035cd0243398d1e294066f0e9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.926491 20258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe780069d490455e9dc46384034562c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.929101 20334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81ee341c3274675b9ce0eb4f2093272. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.930382 20262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c4b3bbc0854317a9a279a899bf01a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.933178 20409 tablet_service.cc:1467] Processing CreateTablet for tablet 44c4b3bbc0854317a9a279a899bf01a7 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20250112 20:59:25.934410 20409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c4b3bbc0854317a9a279a899bf01a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.934259 20336 tablet_service.cc:1467] Processing CreateTablet for tablet daeaecb620f744ba85b6cd38a83603d9 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:25.935432 20336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daeaecb620f744ba85b6cd38a83603d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.939818 20259 tablet_service.cc:1467] Processing CreateTablet for tablet e81ee341c3274675b9ce0eb4f2093272 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:25.941030 20259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81ee341c3274675b9ce0eb4f2093272. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.941102 20333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910996064780419b92c4e87697377d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.958492 20412 tablet_service.cc:1467] Processing CreateTablet for tablet e81ee341c3274675b9ce0eb4f2093272 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20250112 20:59:25.959792 20412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81ee341c3274675b9ce0eb4f2093272. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.961879 20261 tablet_service.cc:1467] Processing CreateTablet for tablet 910996064780419b92c4e87697377d1a (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:25.963016 20261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910996064780419b92c4e87697377d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.965991 20263 tablet_service.cc:1467] Processing CreateTablet for tablet 2f602ef67f28479caf359326135a508d (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:25.967180 20263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f602ef67f28479caf359326135a508d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.969779 20256 tablet_service.cc:1467] Processing CreateTablet for tablet daeaecb620f744ba85b6cd38a83603d9 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:25.971052 20256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daeaecb620f744ba85b6cd38a83603d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.994086 20410 tablet_service.cc:1467] Processing CreateTablet for tablet 910996064780419b92c4e87697377d1a (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20250112 20:59:25.994731 20404 tablet_service.cc:1467] Processing CreateTablet for tablet daeaecb620f744ba85b6cd38a83603d9 (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20250112 20:59:25.995383 20410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910996064780419b92c4e87697377d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.996994 20404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daeaecb620f744ba85b6cd38a83603d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:25.997606 20408 tablet_service.cc:1467] Processing CreateTablet for tablet 2f602ef67f28479caf359326135a508d (DEFAULT_TABLE table=soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20250112 20:59:25.998844 20408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f602ef67f28479caf359326135a508d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250112 20:59:26.078392 20563 tablet_bootstrap.cc:492] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.085553 20562 tablet_bootstrap.cc:492] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.089386 20564 tablet_bootstrap.cc:492] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.091495 20562 tablet_bootstrap.cc:654] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.098297 20564 tablet_bootstrap.cc:654] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.098805 20563 tablet_bootstrap.cc:654] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.108800 20562 tablet_bootstrap.cc:492] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:26.109894 20563 tablet_bootstrap.cc:492] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.110154 20562 ts_tablet_manager.cc:1397] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.010s
I20250112 20:59:26.110338 20563 ts_tablet_manager.cc:1397] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.032s user 0.012s sys 0.004s
I20250112 20:59:26.113195 20562 raft_consensus.cc:357] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.114420 20562 raft_consensus.cc:383] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.114158 20563 raft_consensus.cc:357] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.114713 20562 raft_consensus.cc:738] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.114882 20563 raft_consensus.cc:383] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.115212 20563 raft_consensus.cc:738] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.115478 20564 tablet_bootstrap.cc:492] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.116025 20564 ts_tablet_manager.cc:1397] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.010s
I20250112 20:59:26.115537 20562 consensus_queue.cc:260] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.115869 20563 consensus_queue.cc:260] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.118443 20563 ts_tablet_manager.cc:1428] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250112 20:59:26.120734 20562 ts_tablet_manager.cc:1428] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20250112 20:59:26.121728 20562 tablet_bootstrap.cc:492] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.122254 20564 raft_consensus.cc:357] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.123077 20564 raft_consensus.cc:383] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.123354 20564 raft_consensus.cc:738] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.120734 20563 tablet_bootstrap.cc:492] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.123976 20564 consensus_queue.cc:260] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.126485 20564 ts_tablet_manager.cc:1428] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.010s user 0.003s sys 0.004s
I20250112 20:59:26.127380 20564 tablet_bootstrap.cc:492] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.130503 20563 tablet_bootstrap.cc:654] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.133296 20564 tablet_bootstrap.cc:654] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.138401 20562 tablet_bootstrap.cc:654] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.144419 20563 tablet_bootstrap.cc:492] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.144706 20564 tablet_bootstrap.cc:492] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.144881 20563 ts_tablet_manager.cc:1397] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.004s
I20250112 20:59:26.145315 20564 ts_tablet_manager.cc:1397] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.012s
I20250112 20:59:26.147428 20563 raft_consensus.cc:357] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.147861 20562 tablet_bootstrap.cc:492] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:26.147992 20563 raft_consensus.cc:383] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.148571 20563 raft_consensus.cc:738] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.147956 20564 raft_consensus.cc:357] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.148686 20562 ts_tablet_manager.cc:1397] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.010s
I20250112 20:59:26.149027 20564 raft_consensus.cc:383] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.149348 20564 raft_consensus.cc:738] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.149286 20563 consensus_queue.cc:260] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.150162 20564 consensus_queue.cc:260] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.151636 20563 ts_tablet_manager.cc:1428] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20250112 20:59:26.151732 20562 raft_consensus.cc:357] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.152484 20564 ts_tablet_manager.cc:1428] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250112 20:59:26.152511 20563 tablet_bootstrap.cc:492] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.152487 20562 raft_consensus.cc:383] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.153223 20562 raft_consensus.cc:738] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.153735 20564 tablet_bootstrap.cc:492] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.153890 20562 consensus_queue.cc:260] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.156448 20562 ts_tablet_manager.cc:1428] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.007s user 0.001s sys 0.004s
I20250112 20:59:26.157361 20562 tablet_bootstrap.cc:492] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.159679 20563 tablet_bootstrap.cc:654] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.162966 20564 tablet_bootstrap.cc:654] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.163388 20562 tablet_bootstrap.cc:654] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.171041 20563 tablet_bootstrap.cc:492] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.171080 20545 raft_consensus.cc:491] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.171568 20563 ts_tablet_manager.cc:1397] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.009s
I20250112 20:59:26.171629 20545 raft_consensus.cc:513] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.173743 20545 leader_election.cc:290] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.174281 20563 raft_consensus.cc:357] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.174976 20563 raft_consensus.cc:383] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.175223 20563 raft_consensus.cc:738] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.175511 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe780069d490455e9dc46384034562c3" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.175899 20563 consensus_queue.cc:260] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.177502 20545 raft_consensus.cc:491] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250112 20:59:26.177199 20237 leader_election.cc:343] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.179522 20545 raft_consensus.cc:513] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.180112 20562 tablet_bootstrap.cc:492] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:26.180372 20563 ts_tablet_manager.cc:1428] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20250112 20:59:26.180553 20562 ts_tablet_manager.cc:1397] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.009s
I20250112 20:59:26.181332 20563 tablet_bootstrap.cc:492] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.181994 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6fc6d0d5b5465da3c14a81f03e98e4" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.182569 20345 raft_consensus.cc:2463] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 0.
I20250112 20:59:26.183338 20562 raft_consensus.cc:357] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.183679 20237 leader_election.cc:304] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:26.178404 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe780069d490455e9dc46384034562c3" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.184489 20550 raft_consensus.cc:2798] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.184057 20562 raft_consensus.cc:383] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.184789 20545 leader_election.cc:290] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.185016 20562 raft_consensus.cc:738] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.184870 20421 raft_consensus.cc:2463] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 0.
I20250112 20:59:26.184924 20550 raft_consensus.cc:491] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.185760 20562 consensus_queue.cc:260] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.186046 20550 raft_consensus.cc:3054] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.186795 20236 leader_election.cc:304] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters: 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.187630 20563 tablet_bootstrap.cc:654] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.187820 20545 raft_consensus.cc:2798] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.188201 20545 raft_consensus.cc:491] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.188607 20564 tablet_bootstrap.cc:492] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.188736 20545 raft_consensus.cc:3054] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.189085 20564 ts_tablet_manager.cc:1397] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.036s user 0.015s sys 0.008s
I20250112 20:59:26.191741 20564 raft_consensus.cc:357] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.192339 20564 raft_consensus.cc:383] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.192577 20564 raft_consensus.cc:738] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.193166 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6fc6d0d5b5465da3c14a81f03e98e4" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.193281 20564 consensus_queue.cc:260] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
W20250112 20:59:26.194794 20236 leader_election.cc:343] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.196563 20562 ts_tablet_manager.cc:1428] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.016s user 0.004s sys 0.002s
I20250112 20:59:26.197491 20562 tablet_bootstrap.cc:492] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.198638 20563 tablet_bootstrap.cc:492] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.199095 20563 ts_tablet_manager.cc:1397] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20250112 20:59:26.200932 20515 raft_consensus.cc:491] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.201366 20515 raft_consensus.cc:513] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.201707 20563 raft_consensus.cc:357] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.202407 20563 raft_consensus.cc:383] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.202718 20563 raft_consensus.cc:738] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.203326 20515 leader_election.cc:290] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.203590 20562 tablet_bootstrap.cc:654] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.203379 20563 consensus_queue.cc:260] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.206913 20563 ts_tablet_manager.cc:1428] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20250112 20:59:26.207737 20563 tablet_bootstrap.cc:492] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.208037 20564 ts_tablet_manager.cc:1428] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20250112 20:59:26.208950 20564 tablet_bootstrap.cc:492] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.209442 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a0ef67bf6496e94624895d52de105" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
W20250112 20:59:26.210995 20311 leader_election.cc:343] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.212313 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a0ef67bf6496e94624895d52de105" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.212693 20550 raft_consensus.cc:513] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
W20250112 20:59:26.213521 20311 leader_election.cc:343] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.213812 20563 tablet_bootstrap.cc:654] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.213927 20311 leader_election.cc:304] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142; no voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.214845 20564 tablet_bootstrap.cc:654] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.215430 20550 leader_election.cc:290] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.218688 20420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6fc6d0d5b5465da3c14a81f03e98e4" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:26.219002 20515 raft_consensus.cc:2743] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:26.219277 20562 tablet_bootstrap.cc:492] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:26.218596 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6fc6d0d5b5465da3c14a81f03e98e4" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:26.219698 20562 ts_tablet_manager.cc:1397] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.000s
I20250112 20:59:26.219944 20345 raft_consensus.cc:3054] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
W20250112 20:59:26.220011 20236 leader_election.cc:343] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.221860 20564 tablet_bootstrap.cc:492] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.222314 20564 ts_tablet_manager.cc:1397] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20250112 20:59:26.222147 20562 raft_consensus.cc:357] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.222857 20562 raft_consensus.cc:383] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.223093 20562 raft_consensus.cc:738] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.223592 20563 tablet_bootstrap.cc:492] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.224078 20563 ts_tablet_manager.cc:1397] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20250112 20:59:26.223709 20562 consensus_queue.cc:260] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.224946 20564 raft_consensus.cc:357] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.225705 20564 raft_consensus.cc:383] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.226013 20564 raft_consensus.cc:738] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.226174 20562 ts_tablet_manager.cc:1428] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20250112 20:59:26.227033 20562 tablet_bootstrap.cc:492] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.226691 20564 consensus_queue.cc:260] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.227082 20563 raft_consensus.cc:357] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.228340 20563 raft_consensus.cc:383] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.228709 20563 raft_consensus.cc:738] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.229413 20563 consensus_queue.cc:260] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.230549 20561 raft_consensus.cc:491] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.230949 20561 raft_consensus.cc:513] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.232826 20561 leader_election.cc:290] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.236845 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c4b3bbc0854317a9a279a899bf01a7" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:26.238359 20420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c4b3bbc0854317a9a279a899bf01a7" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.239068 20420 raft_consensus.cc:2463] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 0.
I20250112 20:59:26.239323 20562 tablet_bootstrap.cc:654] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.240419 20311 leader_election.cc:304] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
W20250112 20:59:26.241384 20311 leader_election.cc:343] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.241863 20561 raft_consensus.cc:2798] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.242391 20561 raft_consensus.cc:491] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.242717 20561 raft_consensus.cc:3054] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.243738 20563 ts_tablet_manager.cc:1428] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.019s user 0.008s sys 0.004s
I20250112 20:59:26.244239 20564 ts_tablet_manager.cc:1428] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.022s user 0.006s sys 0.003s
I20250112 20:59:26.244920 20563 tablet_bootstrap.cc:492] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.249922 20561 raft_consensus.cc:513] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.247527 20345 raft_consensus.cc:2463] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 1.
I20250112 20:59:26.252439 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c4b3bbc0854317a9a279a899bf01a7" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:26.249053 20545 raft_consensus.cc:513] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.253717 20564 tablet_bootstrap.cc:492] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.253991 20237 leader_election.cc:304] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters: 0c62e542bb824359a6a7a83070a62b61
I20250112 20:59:26.255100 20550 raft_consensus.cc:2798] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.255582 20550 raft_consensus.cc:695] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
W20250112 20:59:26.255671 20311 leader_election.cc:343] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.256527 20550 consensus_queue.cc:237] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.257876 20561 leader_election.cc:290] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.258033 20420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c4b3bbc0854317a9a279a899bf01a7" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:26.258662 20420 raft_consensus.cc:3054] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.260802 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe780069d490455e9dc46384034562c3" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:26.261412 20421 raft_consensus.cc:3054] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.264813 20564 tablet_bootstrap.cc:654] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.264971 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe780069d490455e9dc46384034562c3" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
W20250112 20:59:26.266216 20237 leader_election.cc:343] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.268913 20420 raft_consensus.cc:2463] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 1.
I20250112 20:59:26.272048 20421 raft_consensus.cc:2463] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 1.
I20250112 20:59:26.273028 20311 leader_election.cc:304] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters: 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.270681 20563 tablet_bootstrap.cc:654] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.273780 20545 leader_election.cc:290] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.273852 20561 raft_consensus.cc:2798] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.274626 20561 raft_consensus.cc:695] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:26.275508 20561 consensus_queue.cc:237] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.278137 20564 tablet_bootstrap.cc:492] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.278695 20564 ts_tablet_manager.cc:1397] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.004s
I20250112 20:59:26.280949 20562 tablet_bootstrap.cc:492] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
W20250112 20:59:26.281116 20219 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20250112 20:59:26.281463 20562 ts_tablet_manager.cc:1397] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.055s user 0.013s sys 0.012s
I20250112 20:59:26.281714 20564 raft_consensus.cc:357] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.282467 20564 raft_consensus.cc:383] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.283653 20236 leader_election.cc:304] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters: 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.284770 20550 raft_consensus.cc:2798] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.285219 20550 raft_consensus.cc:695] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:26.285461 20168 catalog_manager.cc:5526] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:26.286059 20550 consensus_queue.cc:237] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.289662 20563 tablet_bootstrap.cc:492] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.289436 20562 raft_consensus.cc:357] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.290221 20562 raft_consensus.cc:383] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.290340 20563 ts_tablet_manager.cc:1397] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.046s user 0.011s sys 0.008s
I20250112 20:59:26.290544 20562 raft_consensus.cc:738] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.291321 20562 consensus_queue.cc:260] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.293352 20563 raft_consensus.cc:357] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.294211 20563 raft_consensus.cc:383] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.294528 20563 raft_consensus.cc:738] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.295240 20563 consensus_queue.cc:260] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.297606 20563 ts_tablet_manager.cc:1428] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20250112 20:59:26.298606 20563 tablet_bootstrap.cc:492] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.301124 20549 raft_consensus.cc:491] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.301677 20549 raft_consensus.cc:513] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.303685 20549 leader_election.cc:290] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:26.304286 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81ee341c3274675b9ce0eb4f2093272" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:26.282761 20564 raft_consensus.cc:738] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.306310 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81ee341c3274675b9ce0eb4f2093272" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.306821 20563 tablet_bootstrap.cc:654] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.307049 20168 catalog_manager.cc:5526] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
W20250112 20:59:26.307641 20386 leader_election.cc:343] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.308475 20169 catalog_manager.cc:5526] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:26.309927 20562 ts_tablet_manager.cc:1428] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.028s user 0.008s sys 0.000s
I20250112 20:59:26.309756 20564 consensus_queue.cc:260] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.310668 20271 raft_consensus.cc:2463] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
I20250112 20:59:26.311008 20562 tablet_bootstrap.cc:492] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.312819 20385 leader_election.cc:304] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters: 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.313854 20549 raft_consensus.cc:2798] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.314189 20549 raft_consensus.cc:491] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.314481 20549 raft_consensus.cc:3054] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.316583 20562 tablet_bootstrap.cc:654] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.322654 20549 raft_consensus.cc:513] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.325412 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81ee341c3274675b9ce0eb4f2093272" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:26.325776 20562 tablet_bootstrap.cc:492] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:26.325981 20271 raft_consensus.cc:3054] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.327390 20564 ts_tablet_manager.cc:1428] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.048s user 0.009s sys 0.000s
I20250112 20:59:26.328446 20564 tablet_bootstrap.cc:492] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.330011 20562 ts_tablet_manager.cc:1397] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.005s
I20250112 20:59:26.331502 20563 tablet_bootstrap.cc:492] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.332089 20563 ts_tablet_manager.cc:1397] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.034s user 0.009s sys 0.001s
I20250112 20:59:26.333691 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81ee341c3274675b9ce0eb4f2093272" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
W20250112 20:59:26.334863 20386 leader_election.cc:343] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.336381 20564 tablet_bootstrap.cc:654] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.336346 20563 raft_consensus.cc:357] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.337188 20563 raft_consensus.cc:383] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.337221 20271 raft_consensus.cc:2463] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:26.337549 20563 raft_consensus.cc:738] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.338289 20563 consensus_queue.cc:260] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.338403 20562 raft_consensus.cc:357] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.338640 20385 leader_election.cc:304] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters: 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.339130 20562 raft_consensus.cc:383] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.339272 20549 leader_election.cc:290] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:26.339813 20549 raft_consensus.cc:2798] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.339479 20562 raft_consensus.cc:738] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.340482 20549 raft_consensus.cc:695] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:26.341665 20549 consensus_queue.cc:237] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.341609 20562 consensus_queue.cc:260] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.348536 20563 ts_tablet_manager.cc:1428] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20250112 20:59:26.350612 20563 tablet_bootstrap.cc:492] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.356393 20563 tablet_bootstrap.cc:654] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.374166 20170 catalog_manager.cc:5526] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:26.375751 20563 tablet_bootstrap.cc:492] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.376269 20563 ts_tablet_manager.cc:1397] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.000s
I20250112 20:59:26.378646 20563 raft_consensus.cc:357] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.379319 20563 raft_consensus.cc:383] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.379550 20563 raft_consensus.cc:738] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.379897 20564 tablet_bootstrap.cc:492] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.380404 20564 ts_tablet_manager.cc:1397] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.052s user 0.032s sys 0.014s
I20250112 20:59:26.380133 20563 consensus_queue.cc:260] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.382390 20563 ts_tablet_manager.cc:1428] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250112 20:59:26.383525 20563 tablet_bootstrap.cc:492] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142: Bootstrap starting.
I20250112 20:59:26.383301 20564 raft_consensus.cc:357] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.384197 20564 raft_consensus.cc:383] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.384536 20564 raft_consensus.cc:738] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.388823 20563 tablet_bootstrap.cc:654] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.393527 20563 tablet_bootstrap.cc:492] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142: No bootstrap required, opened a new log
I20250112 20:59:26.393908 20563 ts_tablet_manager.cc:1397] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20250112 20:59:26.393827 20564 consensus_queue.cc:260] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.396354 20563 raft_consensus.cc:357] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.397203 20563 raft_consensus.cc:383] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.397477 20563 raft_consensus.cc:738] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.398138 20563 consensus_queue.cc:260] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.399441 20564 ts_tablet_manager.cc:1428] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20250112 20:59:26.400355 20563 ts_tablet_manager.cc:1428] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250112 20:59:26.402287 20561 raft_consensus.cc:491] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.404534 20561 raft_consensus.cc:513] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.405282 20564 tablet_bootstrap.cc:492] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.406390 20561 leader_election.cc:290] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.408205 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcf8035cd0243398d1e294066f0e9b0" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:26.408967 20271 raft_consensus.cc:2463] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 0.
I20250112 20:59:26.409960 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcf8035cd0243398d1e294066f0e9b0" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.410578 20421 raft_consensus.cc:2463] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 0.
I20250112 20:59:26.411842 20311 leader_election.cc:304] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:26.412678 20564 tablet_bootstrap.cc:654] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.413540 20579 raft_consensus.cc:2798] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.413944 20579 raft_consensus.cc:491] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.414242 20579 raft_consensus.cc:3054] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.400682 20562 ts_tablet_manager.cc:1428] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.070s user 0.009s sys 0.005s
I20250112 20:59:26.429066 20562 tablet_bootstrap.cc:492] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.430785 20550 raft_consensus.cc:491] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.430925 20579 raft_consensus.cc:513] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.431293 20550 raft_consensus.cc:513] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.433414 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcf8035cd0243398d1e294066f0e9b0" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:26.433696 20550 leader_election.cc:290] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.433981 20271 raft_consensus.cc:3054] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.435137 20562 tablet_bootstrap.cc:654] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.437112 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f602ef67f28479caf359326135a508d" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
W20250112 20:59:26.438455 20236 leader_election.cc:343] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.439687 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f602ef67f28479caf359326135a508d" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.440666 20345 raft_consensus.cc:2463] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 0.
I20250112 20:59:26.441767 20237 leader_election.cc:304] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters: 0c62e542bb824359a6a7a83070a62b61
I20250112 20:59:26.442793 20550 raft_consensus.cc:2798] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.443120 20550 raft_consensus.cc:491] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.443436 20550 raft_consensus.cc:3054] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.445214 20579 leader_election.cc:290] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.446206 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcf8035cd0243398d1e294066f0e9b0" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:26.446761 20421 raft_consensus.cc:3054] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.448290 20271 raft_consensus.cc:2463] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 1.
I20250112 20:59:26.449640 20311 leader_election.cc:304] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:26.450031 20562 tablet_bootstrap.cc:492] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:26.450526 20562 ts_tablet_manager.cc:1397] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.007s
I20250112 20:59:26.451007 20564 tablet_bootstrap.cc:492] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.451505 20564 ts_tablet_manager.cc:1397] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.047s user 0.016s sys 0.014s
I20250112 20:59:26.453224 20562 raft_consensus.cc:357] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.454056 20562 raft_consensus.cc:383] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.454409 20562 raft_consensus.cc:738] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.454432 20421 raft_consensus.cc:2463] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 1.
I20250112 20:59:26.454243 20564 raft_consensus.cc:357] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.455263 20579 raft_consensus.cc:2798] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.455334 20564 raft_consensus.cc:383] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.455866 20564 raft_consensus.cc:738] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.455348 20562 consensus_queue.cc:260] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.456810 20579 raft_consensus.cc:695] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:26.457595 20579 consensus_queue.cc:237] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.458365 20562 ts_tablet_manager.cc:1428] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20250112 20:59:26.459348 20562 tablet_bootstrap.cc:492] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.456506 20564 consensus_queue.cc:260] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.460740 20564 ts_tablet_manager.cc:1428] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250112 20:59:26.461338 20590 raft_consensus.cc:491] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.461849 20590 raft_consensus.cc:513] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.463742 20590 leader_election.cc:290] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:26.466919 20550 raft_consensus.cc:513] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.467382 20345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910996064780419b92c4e87697377d1a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.468887 20562 tablet_bootstrap.cc:654] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.469735 20550 leader_election.cc:290] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.470476 20564 tablet_bootstrap.cc:492] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.470505 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f602ef67f28479caf359326135a508d" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:26.473277 20348 raft_consensus.cc:3054] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.473316 20168 catalog_manager.cc:5526] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 1 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:26.475838 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910996064780419b92c4e87697377d1a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:26.468914 20345 raft_consensus.cc:2463] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 0.
W20250112 20:59:26.477144 20385 leader_election.cc:343] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.477771 20386 leader_election.cc:304] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters: 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.478614 20590 raft_consensus.cc:2798] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.478957 20590 raft_consensus.cc:491] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.479298 20590 raft_consensus.cc:3054] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.479809 20564 tablet_bootstrap.cc:654] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.483156 20348 raft_consensus.cc:2463] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 1.
I20250112 20:59:26.484208 20237 leader_election.cc:304] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:26.484939 20550 raft_consensus.cc:2798] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.485327 20550 raft_consensus.cc:695] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:26.486048 20550 consensus_queue.cc:237] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.486938 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f602ef67f28479caf359326135a508d" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:26.487051 20590 raft_consensus.cc:513] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
W20250112 20:59:26.488583 20236 leader_election.cc:343] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.489609 20590 leader_election.cc:290] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:26.491039 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910996064780419b92c4e87697377d1a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96"
W20250112 20:59:26.492445 20385 leader_election.cc:343] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Illegal state: must be running to vote when last-logged opid is not known
I20250112 20:59:26.493566 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910996064780419b92c4e87697377d1a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:26.494179 20346 raft_consensus.cc:3054] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.498366 20562 tablet_bootstrap.cc:492] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:26.498807 20562 ts_tablet_manager.cc:1397] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.011s
I20250112 20:59:26.501807 20562 raft_consensus.cc:357] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.502575 20562 raft_consensus.cc:383] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.502893 20562 raft_consensus.cc:738] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.503683 20562 consensus_queue.cc:260] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.508248 20562 ts_tablet_manager.cc:1428] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.009s user 0.007s sys 0.002s
I20250112 20:59:26.509131 20562 tablet_bootstrap.cc:492] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61: Bootstrap starting.
I20250112 20:59:26.509351 20346 raft_consensus.cc:2463] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 1.
I20250112 20:59:26.511437 20564 tablet_bootstrap.cc:492] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.512002 20564 ts_tablet_manager.cc:1397] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.042s user 0.021s sys 0.014s
I20250112 20:59:26.512275 20386 leader_election.cc:304] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters: 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.513096 20590 raft_consensus.cc:2798] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.513532 20590 raft_consensus.cc:695] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:26.514231 20564 raft_consensus.cc:357] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.514266 20590 consensus_queue.cc:237] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.514096 20170 catalog_manager.cc:5526] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:26.515022 20564 raft_consensus.cc:383] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.515298 20564 raft_consensus.cc:738] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.515763 20562 tablet_bootstrap.cc:654] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.515841 20564 consensus_queue.cc:260] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.518424 20564 ts_tablet_manager.cc:1428] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20250112 20:59:26.519486 20564 tablet_bootstrap.cc:492] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96: Bootstrap starting.
I20250112 20:59:26.524725 20550 raft_consensus.cc:491] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.525331 20550 raft_consensus.cc:513] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.526360 20564 tablet_bootstrap.cc:654] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96: Neither blocks nor log segments found. Creating new log.
I20250112 20:59:26.526504 20562 tablet_bootstrap.cc:492] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61: No bootstrap required, opened a new log
I20250112 20:59:26.526986 20562 ts_tablet_manager.cc:1397] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20250112 20:59:26.527170 20550 leader_election.cc:290] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.529068 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910996064780419b92c4e87697377d1a" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.529827 20346 raft_consensus.cc:2388] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 792c9d15fc44406b9058909c5a860f96 in current term 1: Already voted for candidate 0c62e542bb824359a6a7a83070a62b61 in this term.
I20250112 20:59:26.529556 20562 raft_consensus.cc:357] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.530208 20562 raft_consensus.cc:383] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.530457 20562 raft_consensus.cc:738] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.531031 20562 consensus_queue.cc:260] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.531556 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910996064780419b92c4e87697377d1a" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.533126 20562 ts_tablet_manager.cc:1428] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20250112 20:59:26.533491 20236 leader_election.cc:304] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792c9d15fc44406b9058909c5a860f96; no voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.534788 20550 raft_consensus.cc:3054] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.536135 20564 tablet_bootstrap.cc:492] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96: No bootstrap required, opened a new log
I20250112 20:59:26.536542 20564 ts_tablet_manager.cc:1397] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20250112 20:59:26.536803 20545 raft_consensus.cc:491] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.537374 20545 raft_consensus.cc:513] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.538975 20564 raft_consensus.cc:357] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.539636 20564 raft_consensus.cc:383] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250112 20:59:26.539832 20545 leader_election.cc:290] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.539888 20564 raft_consensus.cc:738] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Initialized, Role: FOLLOWER
I20250112 20:59:26.540838 20564 consensus_queue.cc:260] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.541833 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a0ef67bf6496e94624895d52de105" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.542517 20346 raft_consensus.cc:2463] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 0.
I20250112 20:59:26.542912 20564 ts_tablet_manager.cc:1428] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250112 20:59:26.543828 20237 leader_election.cc:304] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:26.545238 20575 raft_consensus.cc:2798] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.545563 20575 raft_consensus.cc:491] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.545915 20575 raft_consensus.cc:3054] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.545944 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a0ef67bf6496e94624895d52de105" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.547058 20421 raft_consensus.cc:2463] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 0.
I20250112 20:59:26.551175 20575 raft_consensus.cc:513] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.553290 20575 leader_election.cc:290] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.553479 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a0ef67bf6496e94624895d52de105" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:26.554168 20421 raft_consensus.cc:3054] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.554339 20550 raft_consensus.cc:2743] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:26.556259 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be0a0ef67bf6496e94624895d52de105" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:26.556823 20346 raft_consensus.cc:3054] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.559618 20421 raft_consensus.cc:2463] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 1.
I20250112 20:59:26.560547 20236 leader_election.cc:304] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:26.561632 20550 raft_consensus.cc:2798] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.562031 20550 raft_consensus.cc:695] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:26.562713 20550 consensus_queue.cc:237] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.562870 20346 raft_consensus.cc:2463] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 1.
I20250112 20:59:26.564541 20170 catalog_manager.cc:5526] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 1 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:26.570119 20169 catalog_manager.cc:5526] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:26.690421 20579 raft_consensus.cc:491] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.690907 20579 raft_consensus.cc:513] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.692497 20579 leader_election.cc:290] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.693189 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe780069d490455e9dc46384034562c3" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:26.693478 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe780069d490455e9dc46384034562c3" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.694203 20421 raft_consensus.cc:2388] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a2063dd1fee4cb48fe32bf917055142 in current term 1: Already voted for candidate 792c9d15fc44406b9058909c5a860f96 in this term.
I20250112 20:59:26.694641 20550 raft_consensus.cc:491] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.694981 20311 leader_election.cc:304] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142; no voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.695084 20550 raft_consensus.cc:513] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.695748 20579 raft_consensus.cc:3054] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.696684 20550 leader_election.cc:290] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.697561 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeaecb620f744ba85b6cd38a83603d9" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.697618 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeaecb620f744ba85b6cd38a83603d9" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.698132 20346 raft_consensus.cc:2463] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 0.
I20250112 20:59:26.698334 20421 raft_consensus.cc:2463] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 0.
I20250112 20:59:26.698985 20237 leader_election.cc:304] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:26.699648 20550 raft_consensus.cc:2798] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250112 20:59:26.699896 20550 raft_consensus.cc:491] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250112 20:59:26.700119 20550 raft_consensus.cc:3054] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.701498 20579 raft_consensus.cc:2743] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:26.704149 20550 raft_consensus.cc:513] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.705724 20550 leader_election.cc:290] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.706259 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeaecb620f744ba85b6cd38a83603d9" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:26.706566 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeaecb620f744ba85b6cd38a83603d9" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:26.706732 20346 raft_consensus.cc:3054] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.707023 20421 raft_consensus.cc:3054] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.711761 20346 raft_consensus.cc:2463] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 1.
I20250112 20:59:26.712052 20421 raft_consensus.cc:2463] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 1.
I20250112 20:59:26.712729 20237 leader_election.cc:304] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:26.713325 20550 raft_consensus.cc:2798] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader election won for term 1
I20250112 20:59:26.713680 20550 raft_consensus.cc:695] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:26.714318 20550 consensus_queue.cc:237] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.721391 20170 catalog_manager.cc:5526] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 1 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:26.724786 20590 raft_consensus.cc:491] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.725227 20590 raft_consensus.cc:513] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.726955 20590 leader_election.cc:290] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:26.727847 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6fc6d0d5b5465da3c14a81f03e98e4" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:26.728266 20579 consensus_queue.cc:1035] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.729671 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6fc6d0d5b5465da3c14a81f03e98e4" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.730477 20346 raft_consensus.cc:2388] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0c62e542bb824359a6a7a83070a62b61 in current term 1: Already voted for candidate 792c9d15fc44406b9058909c5a860f96 in this term.
I20250112 20:59:26.731295 20386 leader_election.cc:304] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61; no voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.732087 20590 raft_consensus.cc:3054] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.738729 20590 raft_consensus.cc:2743] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:26.741737 20271 raft_consensus.cc:3054] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.747848 20550 raft_consensus.cc:491] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.748667 20550 raft_consensus.cc:513] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.748767 20515 consensus_queue.cc:1035] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.752452 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c4b3bbc0854317a9a279a899bf01a7" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.754763 20550 leader_election.cc:290] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.755645 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c4b3bbc0854317a9a279a899bf01a7" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.757138 20236 leader_election.cc:304] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792c9d15fc44406b9058909c5a860f96; no voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.758129 20550 raft_consensus.cc:2743] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:59:26.775647 20590 raft_consensus.cc:491] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.776129 20590 raft_consensus.cc:513] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.777583 20590 leader_election.cc:290] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:26.778321 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f602ef67f28479caf359326135a508d" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:26.778448 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f602ef67f28479caf359326135a508d" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:26.779140 20346 raft_consensus.cc:2388] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0c62e542bb824359a6a7a83070a62b61 in current term 1: Already voted for candidate 792c9d15fc44406b9058909c5a860f96 in this term.
I20250112 20:59:26.779960 20386 leader_election.cc:304] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61; no voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.780504 20590 raft_consensus.cc:3054] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.784842 20590 raft_consensus.cc:2743] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250112 20:59:26.804180 20590 consensus_queue.cc:1035] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.810097 20550 consensus_queue.cc:1035] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.815183 20346 raft_consensus.cc:3054] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 0 FOLLOWER]: Advancing to term 1
I20250112 20:59:26.827339 20588 consensus_queue.cc:1035] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:59:26.829499 20515 raft_consensus.cc:491] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250112 20:59:26.830062 20515 raft_consensus.cc:513] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.831853 20515 leader_election.cc:290] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.833019 20420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81ee341c3274675b9ce0eb4f2093272" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c62e542bb824359a6a7a83070a62b61" is_pre_election: true
I20250112 20:59:26.849449 20346 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0bcf8035cd0243398d1e294066f0e9b0"
dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
mode: GRACEFUL
new_leader_uuid: "792c9d15fc44406b9058909c5a860f96"
from {username='slave'} at 127.0.0.1:54160
I20250112 20:59:26.850242 20346 raft_consensus.cc:604] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Received request to transfer leadership to TS 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.859941 20420 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e81ee341c3274675b9ce0eb4f2093272"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
mode: GRACEFUL
new_leader_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:46570
I20250112 20:59:26.860456 20420 raft_consensus.cc:604] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Received request to transfer leadership to TS 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.860721 20550 consensus_queue.cc:1035] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.861248 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81ee341c3274675b9ce0eb4f2093272" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:26.863384 20550 consensus_queue.cc:1035] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.864723 20311 leader_election.cc:304] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a2063dd1fee4cb48fe32bf917055142; no voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:26.864954 20550 consensus_queue.cc:1035] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.865401 20420 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "910996064780419b92c4e87697377d1a"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
mode: GRACEFUL
new_leader_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:46570
I20250112 20:59:26.865976 20420 raft_consensus.cc:604] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Received request to transfer leadership to TS 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.866287 20608 raft_consensus.cc:2743] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250112 20:59:26.896464 20271 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2f602ef67f28479caf359326135a508d"
dest_uuid: "792c9d15fc44406b9058909c5a860f96"
mode: GRACEFUL
new_leader_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:59078
I20250112 20:59:26.897059 20271 raft_consensus.cc:604] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Received request to transfer leadership to TS 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:26.901788 20271 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "daeaecb620f744ba85b6cd38a83603d9"
dest_uuid: "792c9d15fc44406b9058909c5a860f96"
mode: GRACEFUL
new_leader_uuid: "0c62e542bb824359a6a7a83070a62b61"
from {username='slave'} at 127.0.0.1:59078
I20250112 20:59:26.902323 20271 raft_consensus.cc:604] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Received request to transfer leadership to TS 0c62e542bb824359a6a7a83070a62b61
I20250112 20:59:26.915620 20615 consensus_queue.cc:1035] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.924072 20550 consensus_queue.cc:1035] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.930191 20550 raft_consensus.cc:988] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96: : Instructing follower 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:26.930594 20615 raft_consensus.cc:1076] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Signalling peer 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:26.931913 20346 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2f602ef67f28479caf359326135a508d"
dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:54152
I20250112 20:59:26.932418 20346 raft_consensus.cc:491] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:26.932705 20346 raft_consensus.cc:3054] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:26.937453 20346 raft_consensus.cc:513] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.939097 20346 leader_election.cc:290] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:26.940172 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f602ef67f28479caf359326135a508d" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:26.940083 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f602ef67f28479caf359326135a508d" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:26.940853 20421 raft_consensus.cc:3054] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:26.940903 20271 raft_consensus.cc:3049] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:26.941336 20271 raft_consensus.cc:738] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:26.942031 20271 consensus_queue.cc:260] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.943109 20271 raft_consensus.cc:3054] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:26.947154 20421 raft_consensus.cc:2463] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:26.948141 20311 leader_election.cc:304] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:26.948957 20515 raft_consensus.cc:2798] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:26.949327 20515 raft_consensus.cc:695] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:26.949383 20271 raft_consensus.cc:2463] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:26.950205 20515 consensus_queue.cc:237] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:26.955701 20169 catalog_manager.cc:5526] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 1 to 2, leader changed from 792c9d15fc44406b9058909c5a860f96 (127.15.23.129) to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:26.986045 20588 consensus_queue.cc:1035] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:26.994994 20549 consensus_queue.cc:1035] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:27.000046 20515 consensus_queue.cc:1035] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:59:27.003397 20588 raft_consensus.cc:988] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: : Instructing follower 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:27.003907 20549 raft_consensus.cc:1076] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Signalling peer 2a2063dd1fee4cb48fe32bf917055142 to start an election
I20250112 20:59:27.008201 20346 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "910996064780419b92c4e87697377d1a"
dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
from {username='slave'} at 127.0.0.1:54156
I20250112 20:59:27.008808 20346 raft_consensus.cc:491] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:27.009189 20346 raft_consensus.cc:3054] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:27.013001 20561 consensus_queue.cc:1035] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:27.018918 20346 raft_consensus.cc:513] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.021776 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910996064780419b92c4e87697377d1a" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:27.022282 20346 leader_election.cc:290] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:27.022308 20271 raft_consensus.cc:3054] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:27.023023 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910996064780419b92c4e87697377d1a" candidate_uuid: "2a2063dd1fee4cb48fe32bf917055142" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:27.023931 20421 raft_consensus.cc:3049] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:27.024313 20421 raft_consensus.cc:738] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:27.025136 20421 consensus_queue.cc:260] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.026321 20421 raft_consensus.cc:3054] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:27.031729 20421 raft_consensus.cc:2463] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:27.032258 20271 raft_consensus.cc:2463] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2063dd1fee4cb48fe32bf917055142 in term 2.
I20250112 20:59:27.032761 20515 raft_consensus.cc:988] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142: : Instructing follower 792c9d15fc44406b9058909c5a860f96 to start an election
I20250112 20:59:27.032682 20311 leader_election.cc:304] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:27.033316 20515 raft_consensus.cc:1076] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Signalling peer 792c9d15fc44406b9058909c5a860f96 to start an election
I20250112 20:59:27.033586 20579 raft_consensus.cc:2798] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:27.034750 20579 raft_consensus.cc:695] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Becoming Leader. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:27.034760 20271 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0bcf8035cd0243398d1e294066f0e9b0"
dest_uuid: "792c9d15fc44406b9058909c5a860f96"
from {username='slave'} at 127.0.0.1:45096
I20250112 20:59:27.035746 20271 raft_consensus.cc:491] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:27.036258 20271 raft_consensus.cc:3054] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:27.035717 20579 consensus_queue.cc:237] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.041865 20271 raft_consensus.cc:513] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.043017 20271 leader_election.cc:290] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 election: Requested vote from peers 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853), 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131:46265)
I20250112 20:59:27.042776 20169 catalog_manager.cc:5526] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 reported cstate change: term changed from 1 to 2, leader changed from 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131) to 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130). New cstate: current_term: 2 leader_uuid: "2a2063dd1fee4cb48fe32bf917055142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:27.043807 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcf8035cd0243398d1e294066f0e9b0" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:27.044410 20346 raft_consensus.cc:3049] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:27.044579 20421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcf8035cd0243398d1e294066f0e9b0" candidate_uuid: "792c9d15fc44406b9058909c5a860f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
I20250112 20:59:27.044737 20346 raft_consensus.cc:738] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:27.045169 20421 raft_consensus.cc:3054] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:27.045575 20346 consensus_queue.cc:260] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.046494 20346 raft_consensus.cc:3054] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:27.051097 20421 raft_consensus.cc:2463] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 2.
I20250112 20:59:27.051453 20346 raft_consensus.cc:2463] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792c9d15fc44406b9058909c5a860f96 in term 2.
I20250112 20:59:27.052088 20236 leader_election.cc:304] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters:
I20250112 20:59:27.052979 20615 raft_consensus.cc:2798] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:27.053328 20615 raft_consensus.cc:695] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Becoming Leader. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:27.053933 20615 consensus_queue.cc:237] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.059317 20169 catalog_manager.cc:5526] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 reported cstate change: term changed from 1 to 2, leader changed from 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130) to 792c9d15fc44406b9058909c5a860f96 (127.15.23.129). New cstate: current_term: 2 leader_uuid: "792c9d15fc44406b9058909c5a860f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: UNKNOWN } } }
I20250112 20:59:27.078024 20615 consensus_queue.cc:1035] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:27.086617 20615 consensus_queue.cc:1035] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250112 20:59:27.134840 20615 consensus_queue.cc:1035] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:27.140166 20614 raft_consensus.cc:988] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96: : Instructing follower 0c62e542bb824359a6a7a83070a62b61 to start an election
I20250112 20:59:27.140671 20613 raft_consensus.cc:1076] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Signalling peer 0c62e542bb824359a6a7a83070a62b61 to start an election
I20250112 20:59:27.142063 20420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "daeaecb620f744ba85b6cd38a83603d9"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
from {username='slave'} at 127.0.0.1:46544
I20250112 20:59:27.142596 20420 raft_consensus.cc:491] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:27.142926 20420 raft_consensus.cc:3054] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:27.146050 20613 consensus_queue.cc:1035] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250112 20:59:27.149524 20420 raft_consensus.cc:513] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.151528 20420 leader_election.cc:290] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 2 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:27.153685 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeaecb620f744ba85b6cd38a83603d9" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:27.154366 20271 raft_consensus.cc:3049] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Stepping down as leader of term 1
I20250112 20:59:27.153079 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daeaecb620f744ba85b6cd38a83603d9" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:27.154667 20421 raft_consensus.cc:1235] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Rejecting Update request from peer 792c9d15fc44406b9058909c5a860f96 for earlier term 1. Current term is 2. Ops: []
I20250112 20:59:27.154816 20271 raft_consensus.cc:738] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:27.155594 20271 consensus_queue.cc:260] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.156441 20271 raft_consensus.cc:3054] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:27.160979 20271 raft_consensus.cc:2463] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 2.
I20250112 20:59:27.161741 20385 leader_election.cc:304] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 792c9d15fc44406b9058909c5a860f96; no voters: 2a2063dd1fee4cb48fe32bf917055142
I20250112 20:59:27.162353 20588 raft_consensus.cc:2798] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader election won for term 2
I20250112 20:59:27.162765 20588 raft_consensus.cc:695] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 2 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:27.163467 20588 consensus_queue.cc:237] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:27.169606 20169 catalog_manager.cc:5526] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 1 to 2, leader changed from 792c9d15fc44406b9058909c5a860f96 (127.15.23.129) to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 2 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:28.292768 20420 raft_consensus.cc:1270] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:28.294273 20635 consensus_queue.cc:1035] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:28.296969 20421 raft_consensus.cc:1270] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:28.298439 20634 consensus_queue.cc:1035] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:28.301035 20271 raft_consensus.cc:1270] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:28.323391 20271 raft_consensus.cc:1270] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Refusing update from remote peer 0c62e542bb824359a6a7a83070a62b61: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:28.324966 20639 consensus_queue.cc:1035] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:59:28.334203 20273 raft_consensus.cc:1270] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Refusing update from remote peer 2a2063dd1fee4cb48fe32bf917055142: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:28.345788 20418 raft_consensus.cc:1270] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Refusing update from remote peer 792c9d15fc44406b9058909c5a860f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:28.347783 20643 raft_consensus.cc:491] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2a2063dd1fee4cb48fe32bf917055142)
I20250112 20:59:28.348291 20643 raft_consensus.cc:513] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:28.350885 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "792c9d15fc44406b9058909c5a860f96" is_pre_election: true
I20250112 20:59:28.353363 20637 consensus_queue.cc:1035] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:28.354974 20643 leader_election.cc:290] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:28.359081 20346 raft_consensus.cc:1270] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Refusing update from remote peer 792c9d15fc44406b9058909c5a860f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:28.364828 20635 consensus_queue.cc:1035] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20250112 20:59:28.367501 20635 consensus_queue.cc:1035] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:28.380908 20346 raft_consensus.cc:3054] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Advancing to term 2
I20250112 20:59:28.388989 20633 consensus_queue.cc:1035] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:28.393801 20346 raft_consensus.cc:1270] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Refusing update from remote peer 0c62e542bb824359a6a7a83070a62b61: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250112 20:59:28.403164 20347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59266f7526c46dc9dadcb79c5e3981a" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2a2063dd1fee4cb48fe32bf917055142" is_pre_election: true
I20250112 20:59:28.409837 20642 consensus_queue.cc:1035] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250112 20:59:28.412405 20386 leader_election.cc:304] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61; no voters: 2a2063dd1fee4cb48fe32bf917055142, 792c9d15fc44406b9058909c5a860f96
I20250112 20:59:28.413403 20642 raft_consensus.cc:2743] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20250112 20:59:29.218065 20443 debug-util.cc:398] Leaking SignalData structure 0x7b08002355c0 after lost signal to thread 15457
W20250112 20:59:29.219017 20443 debug-util.cc:398] Leaking SignalData structure 0x7b0800221160 after lost signal to thread 20205
W20250112 20:59:29.219965 20443 debug-util.cc:398] Leaking SignalData structure 0x7b08000f98c0 after lost signal to thread 20297
W20250112 20:59:29.220827 20443 debug-util.cc:398] Leaking SignalData structure 0x7b080031a1e0 after lost signal to thread 20372
W20250112 20:59:29.221638 20443 debug-util.cc:398] Leaking SignalData structure 0x7b080009e2e0 after lost signal to thread 20446
I20250112 20:59:30.813824 20168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37374:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20250112 20:59:30.836074 20168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37374:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20250112 20:59:30.883641 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:31.053059 20707 tablet.cc:1721] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.042776 20705 tablet.cc:1721] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.147035 20715 tablet.cc:1721] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250112 20:59:31.142796 20703 log.cc:926] Time spent T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142: Append to log took a long time: real 0.096s user 0.001s sys 0.000s
I20250112 20:59:31.173835 20348 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4"
dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
mode: GRACEFUL
new_leader_uuid: "0c62e542bb824359a6a7a83070a62b61"
from {username='slave'} at 127.0.0.1:34478
I20250112 20:59:31.174760 20348 raft_consensus.cc:604] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Received request to transfer leadership to TS 0c62e542bb824359a6a7a83070a62b61
W20250112 20:59:31.187368 20311 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.23.131:46265, user_credentials={real_user=slave}} blocked reactor thread for 100616us
I20250112 20:59:31.175716 20724 tablet.cc:1721] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.221689 20730 tablet.cc:1721] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.233992 20728 tablet.cc:1721] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.238092 20707 tablet.cc:1721] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.267797 20705 tablet.cc:1721] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.269166 20689 raft_consensus.cc:988] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142: : Instructing follower 0c62e542bb824359a6a7a83070a62b61 to start an election
I20250112 20:59:31.269866 20722 raft_consensus.cc:1076] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Signalling peer 0c62e542bb824359a6a7a83070a62b61 to start an election
I20250112 20:59:31.203763 20725 tablet.cc:1721] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.273923 20723 tablet.cc:1721] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.280301 20732 tablet.cc:1721] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.294133 20420 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
from {username='slave'} at 127.0.0.1:46558
I20250112 20:59:31.298204 20420 raft_consensus.cc:491] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:31.208032 20729 tablet.cc:1721] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.298671 20420 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:31.313730 20707 tablet.cc:1721] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.306187 20420 raft_consensus.cc:513] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:31.315235 20705 tablet.cc:1721] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.315737 20724 tablet.cc:1721] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.316027 20729 tablet.cc:1721] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.317163 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:31.317749 20272 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:31.318599 20420 leader_election.cc:290] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 3 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:31.293067 20733 tablet.cc:1721] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.320797 20346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43aad0e784aa4014b483577c8564bea4" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:31.321314 20346 raft_consensus.cc:3049] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:59:31.321661 20346 raft_consensus.cc:738] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2a2063dd1fee4cb48fe32bf917055142, State: Running, Role: LEADER
I20250112 20:59:31.322299 20346 consensus_queue.cc:260] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:31.323877 20346 raft_consensus.cc:3054] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:31.334616 20346 raft_consensus.cc:2463] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 3.
I20250112 20:59:31.335815 20386 leader_election.cc:304] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:31.339270 20690 raft_consensus.cc:2798] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:59:31.339761 20690 raft_consensus.cc:695] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 3 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:31.340520 20690 consensus_queue.cc:237] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:31.352905 20732 tablet.cc:1721] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.355370 20727 tablet.cc:1721] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
W20250112 20:59:31.364287 20738 log.cc:926] Time spent T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: Append to log took a long time: real 0.072s user 0.001s sys 0.000s
I20250112 20:59:31.366961 20715 tablet.cc:1721] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.376307 20273 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21"
dest_uuid: "792c9d15fc44406b9058909c5a860f96"
mode: GRACEFUL
new_leader_uuid: "0c62e542bb824359a6a7a83070a62b61"
from {username='slave'} at 127.0.0.1:59084
I20250112 20:59:31.377110 20273 raft_consensus.cc:604] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Received request to transfer leadership to TS 0c62e542bb824359a6a7a83070a62b61
I20250112 20:59:31.376366 20705 tablet.cc:1721] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.381071 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20250112 20:59:31.381685 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:31.383431 20732 tablet.cc:1721] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.385437 20272 raft_consensus.cc:2463] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 3.
I20250112 20:59:31.409265 20707 tablet.cc:1721] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.414947 20730 tablet.cc:1721] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.357537 20728 tablet.cc:1721] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.427857 20168 catalog_manager.cc:5526] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 2 to 3, leader changed from 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130) to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 3 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
W20250112 20:59:31.441233 20736 log.cc:926] Time spent T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142: Append to log took a long time: real 0.082s user 0.000s sys 0.000s
I20250112 20:59:31.446830 20730 tablet.cc:1721] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.456398 20728 tablet.cc:1721] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
PRIMARY KEY (key)
) version 1
I20250112 20:59:31.485401 20169 catalog_manager.cc:6071] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6: soft_deleted_table [id=7f4f7246feb84d45baf008de346a9768] alter complete (version 1)
I20250112 20:59:31.671624 20633 raft_consensus.cc:988] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96: : Instructing follower 0c62e542bb824359a6a7a83070a62b61 to start an election
I20250112 20:59:31.672052 20665 raft_consensus.cc:1076] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Signalling peer 0c62e542bb824359a6a7a83070a62b61 to start an election
I20250112 20:59:31.673491 20422 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21"
dest_uuid: "0c62e542bb824359a6a7a83070a62b61"
from {username='slave'} at 127.0.0.1:46544
I20250112 20:59:31.674041 20422 raft_consensus.cc:491] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250112 20:59:31.674463 20422 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:31.681421 20422 raft_consensus.cc:513] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:31.683696 20272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792c9d15fc44406b9058909c5a860f96"
I20250112 20:59:31.683781 20422 leader_election.cc:290] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 3 election: Requested vote from peers 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087), 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853)
I20250112 20:59:31.684342 20272 raft_consensus.cc:3049] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Stepping down as leader of term 2
I20250112 20:59:31.684741 20272 raft_consensus.cc:738] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 792c9d15fc44406b9058909c5a860f96, State: Running, Role: LEADER
I20250112 20:59:31.684762 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a84696c5572a45ca91c6e96514ea3d21" candidate_uuid: "0c62e542bb824359a6a7a83070a62b61" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2a2063dd1fee4cb48fe32bf917055142"
I20250112 20:59:31.685710 20348 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:31.685892 20272 consensus_queue.cc:260] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:31.687299 20272 raft_consensus.cc:3054] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Advancing to term 3
I20250112 20:59:31.690291 20348 raft_consensus.cc:2463] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 3.
I20250112 20:59:31.691968 20386 leader_election.cc:304] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c62e542bb824359a6a7a83070a62b61, 2a2063dd1fee4cb48fe32bf917055142; no voters:
I20250112 20:59:31.692730 20690 raft_consensus.cc:2798] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 3 FOLLOWER]: Leader election won for term 3
I20250112 20:59:31.693284 20690 raft_consensus.cc:695] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 3 LEADER]: Becoming Leader. State: Replica: 0c62e542bb824359a6a7a83070a62b61, State: Running, Role: LEADER
I20250112 20:59:31.694152 20690 consensus_queue.cc:237] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } }
I20250112 20:59:31.700330 20272 raft_consensus.cc:2463] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c62e542bb824359a6a7a83070a62b61 in term 3.
I20250112 20:59:31.702234 20170 catalog_manager.cc:5526] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 reported cstate change: term changed from 2 to 3, leader changed from 792c9d15fc44406b9058909c5a860f96 (127.15.23.129) to 0c62e542bb824359a6a7a83070a62b61 (127.15.23.131). New cstate: current_term: 3 leader_uuid: "0c62e542bb824359a6a7a83070a62b61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c62e542bb824359a6a7a83070a62b61" member_type: VOTER last_known_addr { host: "127.15.23.131" port: 46265 } health_report { overall_health: HEALTHY } } }
I20250112 20:59:31.755286 20272 raft_consensus.cc:1270] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 3 FOLLOWER]: Refusing update from remote peer 0c62e542bb824359a6a7a83070a62b61: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:59:31.756393 20690 consensus_queue.cc:1035] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:59:31.772945 20348 raft_consensus.cc:1270] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 3 FOLLOWER]: Refusing update from remote peer 0c62e542bb824359a6a7a83070a62b61: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:59:31.778610 20694 consensus_queue.cc:1035] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20250112 20:59:32.153760 20272 raft_consensus.cc:1270] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 3 FOLLOWER]: Refusing update from remote peer 0c62e542bb824359a6a7a83070a62b61: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:59:32.154927 20694 consensus_queue.cc:1035] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792c9d15fc44406b9058909c5a860f96" member_type: VOTER last_known_addr { host: "127.15.23.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:59:32.167393 20348 raft_consensus.cc:1270] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 3 FOLLOWER]: Refusing update from remote peer 0c62e542bb824359a6a7a83070a62b61: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250112 20:59:32.169047 20694 consensus_queue.cc:1035] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a2063dd1fee4cb48fe32bf917055142" member_type: VOTER last_known_addr { host: "127.15.23.130" port: 36853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250112 20:59:32.382539 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:32.387461 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:32.387893 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:33.388834 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:33.394861 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:33.395171 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:34.396418 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:34.402899 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:34.403239 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:35.404018 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:35.408469 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:35.408888 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:36.409600 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:36.414517 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:36.414901 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:37.415760 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:37.420068 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:37.420493 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:38.421273 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:38.425679 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:38.426048 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:39.426899 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:39.431682 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:39.432034 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20250112 20:59:40.432724 15454 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20250112 20:59:40.437404 15454 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20250112 20:59:40.437827 15454 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/auto_leader_rebalancer-test.cc:596: Failure
Expected: (leader.second) != (3), actual: 3 vs 3
I20250112 20:59:41.485838 15454 tablet_server.cc:178] TabletServer@127.15.23.129:0 shutting down...
I20250112 20:59:41.519189 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:59:41.519960 15454 tablet_replica.cc:331] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.520996 15454 raft_consensus.cc:2238] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.521624 15454 raft_consensus.cc:2267] T 910996064780419b92c4e87697377d1a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.524068 15454 tablet_replica.cc:331] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.524816 15454 raft_consensus.cc:2238] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.525318 15454 raft_consensus.cc:2267] T daeaecb620f744ba85b6cd38a83603d9 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.527624 15454 tablet_replica.cc:331] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.528263 15454 raft_consensus.cc:2238] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.529495 15454 raft_consensus.cc:2267] T be0a0ef67bf6496e94624895d52de105 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.531548 15454 tablet_replica.cc:331] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.532092 15454 raft_consensus.cc:2238] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.532955 15454 raft_consensus.cc:2267] T 0bcf8035cd0243398d1e294066f0e9b0 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.535115 15454 tablet_replica.cc:331] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.535674 15454 raft_consensus.cc:2238] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Raft consensus shutting down.
W20250112 20:59:41.536767 20311 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
I20250112 20:59:41.536803 15454 raft_consensus.cc:2267] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.539752 15454 tablet_replica.cc:331] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
W20250112 20:59:41.539803 20311 consensus_peers.cc:487] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 -> Peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Couldn't send request to peer 792c9d15fc44406b9058909c5a860f96. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:41.540560 15454 raft_consensus.cc:2238] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.541102 15454 raft_consensus.cc:2267] T 44c4b3bbc0854317a9a279a899bf01a7 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.543035 15454 tablet_replica.cc:331] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.543563 15454 raft_consensus.cc:2238] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.543951 15454 raft_consensus.cc:2267] T 42b0e5bc9e0749c7b060f68f8855a77b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.546036 15454 tablet_replica.cc:331] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.546538 15454 raft_consensus.cc:2238] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.547143 15454 raft_consensus.cc:2267] T 2f602ef67f28479caf359326135a508d P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.550078 15454 tablet_replica.cc:331] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.550632 15454 raft_consensus.cc:2238] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.551052 15454 raft_consensus.cc:2267] T c439b8ab09aa4503a93dc9f98400a681 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.552991 15454 tablet_replica.cc:331] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.553422 15454 raft_consensus.cc:2238] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.553830 15454 raft_consensus.cc:2267] T c59266f7526c46dc9dadcb79c5e3981a P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.555621 15454 tablet_replica.cc:331] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.556191 15454 raft_consensus.cc:2238] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.556957 15454 raft_consensus.cc:2267] T c9181f2ac504418faf8c6291311cd19b P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.559051 15454 tablet_replica.cc:331] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.559597 15454 raft_consensus.cc:2238] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.560047 15454 raft_consensus.cc:2267] T 74fbee6b4295483fa6ff7285ece93901 P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.561998 15454 tablet_replica.cc:331] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.562527 15454 raft_consensus.cc:2238] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.562956 15454 raft_consensus.cc:2267] T a84696c5572a45ca91c6e96514ea3d21 P 792c9d15fc44406b9058909c5a860f96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.565061 15454 tablet_replica.cc:331] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.565532 15454 raft_consensus.cc:2238] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.566251 15454 raft_consensus.cc:2267] T fe780069d490455e9dc46384034562c3 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.568102 15454 tablet_replica.cc:331] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.568527 15454 raft_consensus.cc:2238] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.569175 15454 raft_consensus.cc:2267] T e81ee341c3274675b9ce0eb4f2093272 P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.571168 15454 tablet_replica.cc:331] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.571599 15454 raft_consensus.cc:2238] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.572072 15454 raft_consensus.cc:2267] T 43aad0e784aa4014b483577c8564bea4 P 792c9d15fc44406b9058909c5a860f96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.573961 15454 tablet_replica.cc:331] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.574378 15454 raft_consensus.cc:2238] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.575076 15454 raft_consensus.cc:2267] T 32907ca5566a44f9a3f284ba36e744ae P 792c9d15fc44406b9058909c5a860f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.576962 15454 tablet_replica.cc:331] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96: stopping tablet replica
I20250112 20:59:41.577363 15454 raft_consensus.cc:2238] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.578178 15454 raft_consensus.cc:2267] T 13960ac9c4094cf7b7a843d3a768433b P 792c9d15fc44406b9058909c5a860f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.622339 15454 tablet_server.cc:195] TabletServer@127.15.23.129:0 shutdown complete.
W20250112 20:59:41.630458 20385 consensus_peers.cc:487] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 -> Peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Couldn't send request to peer 792c9d15fc44406b9058909c5a860f96. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:39087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:59:41.641748 20311 consensus_peers.cc:487] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 -> Peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Couldn't send request to peer 792c9d15fc44406b9058909c5a860f96. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:39087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250112 20:59:41.656672 20385 consensus_peers.cc:487] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 -> Peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Couldn't send request to peer 792c9d15fc44406b9058909c5a860f96. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:39087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:41.665680 15454 tablet_server.cc:178] TabletServer@127.15.23.130:0 shutting down...
I20250112 20:59:41.692488 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:59:41.693166 15454 tablet_replica.cc:331] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.693924 15454 raft_consensus.cc:2238] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.695063 15454 raft_consensus.cc:2267] T 910996064780419b92c4e87697377d1a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.697702 15454 tablet_replica.cc:331] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.698334 15454 raft_consensus.cc:2238] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.698769 15454 raft_consensus.cc:2267] T daeaecb620f744ba85b6cd38a83603d9 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.700753 15454 tablet_replica.cc:331] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.701282 15454 raft_consensus.cc:2238] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.701691 15454 raft_consensus.cc:2267] T fe780069d490455e9dc46384034562c3 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.703855 15454 tablet_replica.cc:331] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.704420 15454 raft_consensus.cc:2238] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.704965 15454 raft_consensus.cc:2267] T 0bcf8035cd0243398d1e294066f0e9b0 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.707096 15454 tablet_replica.cc:331] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.707793 15454 raft_consensus.cc:2238] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.708369 15454 raft_consensus.cc:2267] T be0a0ef67bf6496e94624895d52de105 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:41.709089 20386 consensus_peers.cc:487] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 -> Peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Couldn't send request to peer 2a2063dd1fee4cb48fe32bf917055142. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:41.710794 15454 tablet_replica.cc:331] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.711361 15454 raft_consensus.cc:2238] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.711817 15454 raft_consensus.cc:2267] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.713721 15454 tablet_replica.cc:331] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.714162 15454 raft_consensus.cc:2238] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.714555 15454 raft_consensus.cc:2267] T e81ee341c3274675b9ce0eb4f2093272 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.716349 15454 tablet_replica.cc:331] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.716895 15454 raft_consensus.cc:2238] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.717247 15454 raft_consensus.cc:2267] T 43aad0e784aa4014b483577c8564bea4 P 2a2063dd1fee4cb48fe32bf917055142 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.719046 15454 tablet_replica.cc:331] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.719513 15454 raft_consensus.cc:2238] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.719873 15454 raft_consensus.cc:2267] T 13960ac9c4094cf7b7a843d3a768433b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.721776 15454 tablet_replica.cc:331] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.722239 15454 raft_consensus.cc:2238] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.722694 15454 raft_consensus.cc:2267] T 32907ca5566a44f9a3f284ba36e744ae P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.724592 15454 tablet_replica.cc:331] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.725071 15454 raft_consensus.cc:2238] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.725764 15454 raft_consensus.cc:2267] T 2f602ef67f28479caf359326135a508d P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.727491 15454 tablet_replica.cc:331] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.727967 15454 raft_consensus.cc:2238] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.728389 15454 raft_consensus.cc:2267] T c439b8ab09aa4503a93dc9f98400a681 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.730208 15454 tablet_replica.cc:331] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.730685 15454 raft_consensus.cc:2238] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.731389 15454 raft_consensus.cc:2267] T c59266f7526c46dc9dadcb79c5e3981a P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.733220 15454 tablet_replica.cc:331] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.733826 15454 raft_consensus.cc:2238] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.734766 15454 raft_consensus.cc:2267] T 44c4b3bbc0854317a9a279a899bf01a7 P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:41.736773 20386 consensus_peers.cc:487] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 -> Peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Couldn't send request to peer 2a2063dd1fee4cb48fe32bf917055142. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:41.737162 15454 tablet_replica.cc:331] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.737664 15454 raft_consensus.cc:2238] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.738363 15454 raft_consensus.cc:2267] T 42b0e5bc9e0749c7b060f68f8855a77b P 2a2063dd1fee4cb48fe32bf917055142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.740160 15454 tablet_replica.cc:331] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.740566 15454 raft_consensus.cc:2238] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.741323 15454 raft_consensus.cc:2267] T 74fbee6b4295483fa6ff7285ece93901 P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.743127 15454 tablet_replica.cc:331] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.743567 15454 raft_consensus.cc:2238] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.743955 15454 raft_consensus.cc:2267] T a84696c5572a45ca91c6e96514ea3d21 P 2a2063dd1fee4cb48fe32bf917055142 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.745834 15454 tablet_replica.cc:331] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142: stopping tablet replica
I20250112 20:59:41.746237 15454 raft_consensus.cc:2238] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.746568 15454 raft_consensus.cc:2267] T c9181f2ac504418faf8c6291311cd19b P 2a2063dd1fee4cb48fe32bf917055142 [term 2 FOLLOWER]: Raft consensus is shut down!
W20250112 20:59:41.779819 20386 consensus_peers.cc:487] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 -> Peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Couldn't send request to peer 2a2063dd1fee4cb48fe32bf917055142. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:36853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:41.786423 15454 tablet_server.cc:195] TabletServer@127.15.23.130:0 shutdown complete.
I20250112 20:59:41.820609 15454 tablet_server.cc:178] TabletServer@127.15.23.131:0 shutting down...
W20250112 20:59:41.839218 20386 consensus_peers.cc:487] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 -> Peer 2a2063dd1fee4cb48fe32bf917055142 (127.15.23.130:36853): Couldn't send request to peer 2a2063dd1fee4cb48fe32bf917055142. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.130:36853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:41.846657 15454 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250112 20:59:41.847352 15454 tablet_replica.cc:331] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.848362 15454 raft_consensus.cc:2238] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.849133 15454 raft_consensus.cc:2267] T cc6fc6d0d5b5465da3c14a81f03e98e4 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.851080 15454 tablet_replica.cc:331] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.851504 15454 raft_consensus.cc:2238] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.852182 15454 raft_consensus.cc:2267] T daeaecb620f744ba85b6cd38a83603d9 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.853955 15454 tablet_replica.cc:331] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.854378 15454 raft_consensus.cc:2238] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.854730 15454 raft_consensus.cc:2267] T be0a0ef67bf6496e94624895d52de105 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.856491 15454 tablet_replica.cc:331] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.857039 15454 raft_consensus.cc:2238] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.857528 15454 raft_consensus.cc:2267] T 0bcf8035cd0243398d1e294066f0e9b0 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.859228 15454 tablet_replica.cc:331] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.859709 15454 raft_consensus.cc:2238] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.860183 15454 raft_consensus.cc:2267] T 910996064780419b92c4e87697377d1a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.862563 15454 tablet_replica.cc:331] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.863168 15454 raft_consensus.cc:2238] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus shutting down.
W20250112 20:59:41.863417 20385 consensus_peers.cc:487] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 -> Peer 792c9d15fc44406b9058909c5a860f96 (127.15.23.129:39087): Couldn't send request to peer 792c9d15fc44406b9058909c5a860f96. Status: Network error: Client connection negotiation failed: client connection to 127.15.23.129:39087: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250112 20:59:41.863746 15454 raft_consensus.cc:2267] T 2f602ef67f28479caf359326135a508d P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.865720 15454 tablet_replica.cc:331] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.866176 15454 raft_consensus.cc:2238] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 2 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.866832 15454 raft_consensus.cc:2267] T c439b8ab09aa4503a93dc9f98400a681 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.868567 15454 tablet_replica.cc:331] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.869088 15454 raft_consensus.cc:2238] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.869464 15454 raft_consensus.cc:2267] T fe780069d490455e9dc46384034562c3 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.871203 15454 tablet_replica.cc:331] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.871619 15454 raft_consensus.cc:2238] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.872257 15454 raft_consensus.cc:2267] T e81ee341c3274675b9ce0eb4f2093272 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.873999 15454 tablet_replica.cc:331] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.874433 15454 raft_consensus.cc:2238] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.875111 15454 raft_consensus.cc:2267] T 43aad0e784aa4014b483577c8564bea4 P 0c62e542bb824359a6a7a83070a62b61 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.876751 15454 tablet_replica.cc:331] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.877154 15454 raft_consensus.cc:2238] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.877480 15454 raft_consensus.cc:2267] T c9181f2ac504418faf8c6291311cd19b P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.879033 15454 tablet_replica.cc:331] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.879448 15454 raft_consensus.cc:2238] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.879941 15454 raft_consensus.cc:2267] T c59266f7526c46dc9dadcb79c5e3981a P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.881776 15454 tablet_replica.cc:331] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.882251 15454 raft_consensus.cc:2238] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.882603 15454 raft_consensus.cc:2267] T 13960ac9c4094cf7b7a843d3a768433b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.884444 15454 tablet_replica.cc:331] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.884900 15454 raft_consensus.cc:2238] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.885339 15454 raft_consensus.cc:2267] T 32907ca5566a44f9a3f284ba36e744ae P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.887104 15454 tablet_replica.cc:331] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.887557 15454 raft_consensus.cc:2238] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.887984 15454 raft_consensus.cc:2267] T 44c4b3bbc0854317a9a279a899bf01a7 P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.889746 15454 tablet_replica.cc:331] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.890187 15454 raft_consensus.cc:2238] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.890527 15454 raft_consensus.cc:2267] T 42b0e5bc9e0749c7b060f68f8855a77b P 0c62e542bb824359a6a7a83070a62b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.892320 15454 tablet_replica.cc:331] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.892766 15454 raft_consensus.cc:2238] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 3 LEADER]: Raft consensus shutting down.
I20250112 20:59:41.893487 15454 raft_consensus.cc:2267] T a84696c5572a45ca91c6e96514ea3d21 P 0c62e542bb824359a6a7a83070a62b61 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.895386 15454 tablet_replica.cc:331] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61: stopping tablet replica
I20250112 20:59:41.895800 15454 raft_consensus.cc:2238] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250112 20:59:41.896198 15454 raft_consensus.cc:2267] T 74fbee6b4295483fa6ff7285ece93901 P 0c62e542bb824359a6a7a83070a62b61 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:41.949183 15454 tablet_server.cc:195] TabletServer@127.15.23.131:0 shutdown complete.
I20250112 20:59:41.980409 15454 master.cc:537] Master@127.15.23.190:45649 shutting down...
I20250112 20:59:42.003316 15454 raft_consensus.cc:2238] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 1 LEADER]: Raft consensus shutting down.
I20250112 20:59:42.003855 15454 raft_consensus.cc:2267] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250112 20:59:42.004179 15454 tablet_replica.cc:331] T 00000000000000000000000000000000 P e3919277e93d4feebc3fd09cbc3810e6: stopping tablet replica
I20250112 20:59:42.023698 15454 master.cc:559] Master@127.15.23.190:45649 shutdown complete.
I20250112 20:59:42.051822 15454 test_util.cc:181] -----------------------------------------------
I20250112 20:59:42.052027 15454 test_util.cc:182] Had failures, leaving test files at /tmp/dist-test-task28Uki5/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1736715392595610-15454-0
[ FAILED ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1, where GetParam() = true (20924 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (41594 ms total)
[----------] Global test environment tear-down
[==========] 7 tests from 2 test suites ran. (189303 ms total)
[ PASSED ] 5 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeaderRebalancerTest.TestMaintenanceMode
[ FAILED ] 1 test, listed below:
[ FAILED ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1, where GetParam() = true
1 FAILED TEST
I20250112 20:59:42.058224 15454 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2224 messages since previous log ~598 seconds ago
I20250112 20:59:42.058404 15454 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2224 messages since previous log ~598 seconds ago
I20250112 20:59:42.058537 15454 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2220 messages since previous log ~598 seconds ago
I20250112 20:59:42.058677 15454 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:594: suppressed but not reported on 44 messages since previous log ~57 seconds ago
I20250112 20:59:42.058806 15454 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:542: suppressed but not reported on 44 messages since previous log ~58 seconds ago
I20250112 20:59:42.059005 15454 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 8 messages since previous log ~0 seconds ago
I20250112 20:59:42.059232 15454 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2367: suppressed but not reported on 107 messages since previous log ~40 seconds ago